Lemmy Today

1,493 readers
76 users here now

Welcome to lemmy.today!

About us

🤗 Thanks for joining our little instance here, located in Oregon. The idea is to have a fast, stable instance and allow users to subscribe to whatever content they want from here.

😎 We dont block any other instances. We will keep it that way unless it becomes a moderation problem.

🤠 We will be around for a very long time, so you dont have to worry about us shutting down the instance anytime soon. We like performance and stability in our servers, and will upgrade the instance when its needed.

🥹 Make sure to join a lot of remote communities to get a good feed going. How to do that is explained here.

Lemmy mobile apps

You should start using one of these ASAP since the web browser user interface is quite ugly, even with themes.

Optional Lemmy web browser user interfaces

Rules

Contact the admin

founded 1 year ago
ADMINS
mrmanager
Buelldozer
listing: all - local
search
1
1
X updates its terms to ban crawling and scraping (techcrunch.com)
 
 

cross-posted from: https://kbin.social/m/technology@lemmy.world/t/445789

The new terms, which are effective from September 29, ban any kind of scraping or crawling without “prior written consent.”

NOTE: crawling or scraping the Services in any form, for any purpose without our prior written consent is expressly prohibited.

The previous version of the terms allowed crawling in accordance with robots.txt.

“NOTE: crawling the Services is permissible if done in accordance with the provisions of the robots.txt file, however, scraping the Services without our prior consent is expressly prohibited,” it read.

In the last few months, Twitter has also altered its robots.txt file — a file that gives instructions to robot crawlers about what parts of the site they are permitted to visit — to remove instructions for all crawler bots apart from Google.

In 2015, Twitter confirmed that it had a firehose deal in place with Google to surface tweets in search results. It is not clear if the nature or terms of that deal have changed under the new management.

2
91
X updates its terms to ban crawling and scraping (techcrunch.com)
 
 

The new terms, which are effective from September 29, ban any kind of scraping or crawling without “prior written consent.”

NOTE: crawling or scraping the Services in any form, for any purpose without our prior written consent is expressly prohibited.

The previous version of the terms allowed crawling in accordance with robots.txt.

“NOTE: crawling the Services is permissible if done in accordance with the provisions of the robots.txt file, however, scraping the Services without our prior consent is expressly prohibited,” it read.

In the last few months, Twitter has also altered its robots.txt file — a file that gives instructions to robot crawlers about what parts of the site they are permitted to visit — to remove instructions for all crawler bots apart from Google.

In 2015, Twitter confirmed that it had a firehose deal in place with Google to surface tweets in search results. It is not clear if the nature or terms of that deal have changed under the new management.

3
7
[HN] X updates its terms to ban crawling and scraping (techcrunch.com)
 
 

[ comments | sourced from HackerNews ]

4
2
[TECHCRUNCH] X updates its terms to ban crawling and scraping (techcrunch.com)
 
 

[ sourced from TechCrunch ]

view more: next ›