We can simply add a group of things around the original matrix. So for each element in the original matrix, it is surrounded by 8 numbers and there will be no index problem any longer. When calculating the average, we can simply discard those added things and focus on the real numbers. For example, in my code, I added some -1 to the original matrix.

class Solution(object): def imageSmoother(self, M): """ :type M: List[List[int]] :rtype: List[List[int]] """ N = [[0]*len(M[0]) for i in range(len(M))] M.insert(0,[-1]*(len(M[0])+2)) M.append([-1]*len(M[0])) for i in range(1,len(M)-1): M[i].insert(0,-1) M[i].append(-1) for i in range(1,len(M)-1): for j in range(1,len(M[0])-1): count = 0 Sum = 0 for k in range(-1,2): for l in range(-1,2): if M[i+k][j+l] != -1: count += 1 Sum += M[i+k][j+l] N[i-1][j-1] = int(Sum/count) return N