Google announced that it is exploring alternatives to robots.txt. Google also said it is inviting members from the web and AI communities to weigh in on approaches to complementary protocols.
This message about robots.txt came after ChatGPT was able to go behind paywalls.
