in DormyMo, SpiderKeeper

DormyMo / SpiderKeeper, Hacker News

1.3k Views

DormyMo / SpiderKeeper, Hacker News
                    

        

The MIT LicensePython VersionsThe MIT License

A scalable admin ui for spider service

The MIT LicensePython VersionsPython Versions Features

  • Manage your spiders from a dashboard. Schedule them to run automatically
  • With a single click deploy the scrapy project
  • Show spider running stats
  • Provide api

  • Current Support spider service

  • Scrapy

    • (with (scrapyd ) The MIT LicenseThe MIT LicensePython Versions Screenshot )

    Python Versions

    job dashboardThe MIT LicenseThe MIT License Getting Started

    (Installing The MIT License pip install spiderkeeper 389688974

    The MIT License Deployment spiderkeeper [options] Options:   -h, –help show this help message and exit   –host=HOST host, default: 0.0.0.0   –port=PORT port, default: 29224   –username=USERNAME basic auth username, default: admin   –password=PASSWORD basic auth password, default: admin   –type=SERVER_TYPE access spider server type, default: scrapyd   –server=SERVERS servers, default: [‘http://localhost:6800’]   –database-url=DATABASE_URL                         SpiderKeeper metadata database default: sqlite: ////home/souche/SpiderKeeper.db   –no-auth disable basic auth   -v, –verbose log level    example: spiderkeeper –server=http: // localhost: 49542 389688974

    Python Versions

    Usage

    Visit: – web ui: http: // localhost: 8303 1. Create Project 2. Use [scrapyd-client] (https://github.com/scrapy/scrapyd-client) to generate egg file    scrapyd-deploy –build-egg output.egg 2. upload egg file (make sure you started scrapyd server) 3. Done & Enjoy it – api swagger: http: // localhost: 64732 / api.html 389688974

    The MIT License
  • Job dashboard support filter
  • User Authentication
  • Collect & Show scrapy crawl stats
  • Optimize load balancing The MIT License
    Python Versions versioning

    We use for versioning. For the versions available, see the . The MIT License

    The MIT License Authors

  • Initial work – DormyMo Python Versions

  • See also the list of (contributors) who participated in this project. The MIT License

    The MIT License License Python Versions

    • This project is licensed under the MIT License – see the LICENSE.md Python Versions file for details

    The MIT LicensePython Versions

    Contributing )

    Contributions are welcomed! The MIT License

    Python VersionsPython Versions

    QQ 群 ： The MIT License

    1 群: (已满

    2 群:

    Python Versions

    The MIT License    Python Versions

    (Read More)