Harvesting Mimosa Seed Pods?

When can I harvest my chocolate mimosa pods? I figured that if you can shake the pod and hear the seeds rattle it's OK to pick. When opened the pods have seeds of various colors and sizes due to some of them not drying out completely yet. They range from small and almost black and then from brown to green as they get larger. Will these seed's mature while being stored or should I wait on harvesting them until all the seed's are small and black? Thank you.

Editor's Note The chocolate in the name of the mimosa plant is describing the color. These seeds are toxic and should not be consumed under any circumstances.