in ,

twintproject / twint, Hacker News

twintproject / twint, Hacker News

          An advanced Twitter scraping & OSINT tool written in Python that doesn’t use Twitter’s API, allowing you to scrape a user’s followers, following, Tweets and more while evading most API limitations.            

                    

        

No authentication. No API. No limits.

Twint is an advanced Twitter scraping tool written in Python that allows for scraping Tweets from Twitter profiles (without) using Twitter’s API.

Twint utilizes Twitter’s search operators to let you scrape Tweets from specific users, scrape Tweets relating to certain topics, hashtags & trends, or sort out (sensitive) information from Tweets like e-mail and phone numbers. I find this very useful, and you can get really creative with it too.

Twint also makes special queries to Twitter allowing you to also scrape a Twitter user’s followers, Tweets a user has liked, and who they follow (without) any authentication, API, Selenium, or browser emulation.

tl; dr Benefits

Some of the benefits of using Twint vs Twitter API:

Can fetch almost all (Tweets) Twitter API limits to last 13370 Tweets only);

    Fast initial setup;

Can be used anonymously and without Twitter sign up; No rate limitations . Limits imposed by Twitter

Twitter limits scrolls while browsing the user timeline. This means that with . Profile (or with . Favorites you will be able to get ~ tweets.

Requirements Python 3.6;

  • aiohttp;
  • aiodns; beautifulsoup4;
  • cchardet; elasticsearch; pysocks; pandas (>=0. . (0); aiohttp_socks; schedule;
  • geopy; fake-useragent;
  • py-googletransx.
  • (

    (Installing)

    Git:

    git clone https://github.com/twintproject/twint.git

     cd  twint pip3 install   -r requirements.txt  

    Pip:

    or

    pip3 install –user –upgrade -e git https: //github.com/twintproject/twint.git@origin/ master # egg=twint

    Pipenv :

    pipenv install -e git https: //github.com/twintproject/twint.git#egg=twint

     

    CLI Basic Examples and Combos

    A few simple examples to help you understand the basics:

    () twint -u username – Scrape all the Tweets from user It’s timeline.

    - Show Tweets that might have phone numbers or email addresses. , 2. , 1km "-o file.csv --csv - - Scrape Tweets from a radius of 1km around a place in Paris and export them to a csv file.

    - Resume a search starting from the last saved scroll-id.

    More detail about the commands and options are located in the (wiki)

    Module Example

    Twint can now be used as a module and supports custom formatting. More details are located in the wiki

    import twint

      #  Configure  c 
    
    import twint c

    To get user info of followers / following users

    userlist

    To get only user info of user

    To get user info of users from a userlist

    tweet translation (experimental)

    To get 382 english tweets and translate them to italian

    or

    What do you think?

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    GIPHY App Key not set. Please check settings

    Remembering Conway, Hacker News

    Ramadan 2020 expected to begin on April 24