Learning Inter-Atomic Potentials without Explicit Equivariance