robots.txt
I never really played around with robots.txt- bad bots ignore it anyway so never saw too much of a point. I probably should care what gets indexed by search engines more.
These days with the llm scraping everything though I might as well tell the 'good' ones to go away.
I decided to add the robots.txt entries from this post
hope it works well!