Ok to make sure, The Confederate flag isn't technically a racist symbol right?

I understand that it's a sourthern union thing, not a racist thing, it's just that in those times slavery was accepted. Kinda lke how the union up north calling blacks "N.I.G.G.A" was . So really it isn't racist or am i wrong?