AToM and VXLAN both transport L2 but they do work differently and have different use cases.
AToM is an older protocol, I think from ~2002 and you can use it to transport different L2 protocols over MPLS only. You can transport Ethernet, ATM (AAL5 and cell relay), frame-relay, PPP, and HDLC over MPLS with it. You need to configure it as a pseudowire between two PE routers so it’s a point-to-point link.
VXLAN is a data center technology. In data centers, the 4096 VLANs that 802.1Q offers is way too few. Before virtualization, it used to be 1 physical server = 1 MAC address. Nowadays with virtualization, we run dozens of VMs on a single physical server, trying to isolate those with only 4096 VLANs is a problem. Also, you can run into table issues with ToR (Top-of-Rack) switches that have to store too many MAC addresses in their tables.
VXLAN solves these “L2 issues” by creating an L2 overlay network over IP. It allows you to transport L2 (Ethernet) over an L3 network and has 24 bits so there’s ~16 million VLANs.