Recent searches
Search options
#crawling
The horrifically slow internet speed that provider wants perfectly high amounts of USD for, while they refuse to give us IPv6 address blocks, a required, a highly required service for which they have everything in their infrastructure.
Meet LLMs.txt, a Proposed Standard for AI Website Content Crawling, by @searchengineland.bsky.social:
https://searchengineland.com/llms-txt-proposed-standard-453676
A bit of a back and forth with #Anthropic about my #commercial #rights over their #LLMs, as per my #software #license, that they agreed upon using my code to train their AI.
If this holds, #sourcefirst licenses, or any #foss license with an #open #weights clause, might be the future to prevent the #ai #crawling epidemic.
Make sure your licenses have #teeth, and that they #bite where it hurts the #executives the most. Their wallets.
Please Stop Externalizing Your Costs Directly Into My Face, by @sir:
https://drewdevault.com/2025/03/17/2025-03-17-Stop-externalizing-your-costs-on-me.html
Crawling into the weekend – IG June 2019 https://www.girlselfie.com/628715/crawling-into-the-weekend-ig-june-2019/ #Crawling #june #SydneySweeney #weekend
Google's @methode gives us a partial update on making Google crawling more efficient - there is more work to do... https://www.seroundtable.com/google-more-work-to-do-crawling-38962.html
