There is a way using ffmpeg to add chapter markers to an mp4 after the fact, but it's probably more work than just adding them in FCP and re-exporting. This would be useful if you only have the mp4 and don't have access to the library or the source footage in order to re-export.
I found a good explanation with a sample python script. Tested it and it works.
https://ikyle.me/blog/2020/add-mp4-chapters-ffmpeg
In a nutshell, watch your mp4 and jot the HH:MM:SS times and chapter names into a text file. Then use ffmpeg to extract the existing metadata from the mp4. Run the python script to add your new chapter info into this metadata file. Finally, run ffmpeg to copy this updated metadata file and the mp4 into a new mp4. The new mp4 is not re-encoded, just copied with new metadata.