Simon Vieille 208088416e | ||
---|---|---|
src | ||
.gitignore | ||
README.md | ||
package.json | ||
yarn.lock |
README.md
Scraper
This project is a basic tool to scrap a data from a website using a CSS selector.
For example, if you want to retrieve the number of a project's releases hosted on github:
With CLI
node src/cli.js \
--url https://github.com/foo/bar \
--selector '.repository-content .numbers-summary li:nth-child(4) a' \
--tags \
--breaks \
--spaces \
--breaks \
--trim
...will show XXX releases
.
More help with node src/cli.js --help
.
With code
const scraper = require('deblan-scraper')
const options = {
url: 'https://github.com/foo/bar',
acceptAllStatus: false, // Optional
method: 'GET', // Optional
}
const selector = '.repository-content .numbers-summary li:nth-child(4) a'
const filters = {
tags: null,
breaks: null,
spaces: null,
trim: null,
}
scraper(
options,
selector,
filters,
function(value) {
console.log(value)
},
function(error) {
console.log(error)
}
)
Installation
Requirements:
- node >= 10
- yarn
$ git clone https://gitnet.fr/deblan/scraper.git
$ cd scraper
$ yarn