I am an artist (majored in painting and photography) and have used all kinds of paints extensively.
All acrylic paints are water-based. Acrylic paint is a polymer compound - ie plastic - and drys to an elastic plastic-like state. Once fully dried, the plastic is harmless to ants. Same with oil paints. The oil is the medium used to suspend the pigment. Once fully cured (oils "dry" by a process called oxidation), the paint is also harmless to ants.
What is poisonous to ants (and even humans) are the types of pigments used in the paint. For example, any of the cadmiums are toxic (cadmium red, cadmium yellow, cadmium orange).
If you want to be 100% safe, use off the shelf food coloring. That is what I do. Just dye the pieces you want in food coloring.