Skip to content
/ gopa Public
forked from infinilabs/crawler

GOPA, a spider written in Go. (QQ群号码:333088962)

License

Notifications You must be signed in to change notification settings

lewgun/gopa

 
 

Repository files navigation

Gopa

https://github.com/infinitbyte/gopa

[狗爬], A Spider Written in Go.

Travis Go Report Card Coverage Status

Goal

  • Light weight, low footprint, memory requirement should < 100MB
  • Easy to deploy, no runtime or dependency required
  • Easy to use, no programming or scripts ability needed, out of box features

Build

Mac/Linux: Run make build to build the Gopa

Windows: Checkout this wiki page - How to build GOPA on windows

Download

Release

Start

After download/build the binary file, run ./gopa to start the Gopa

Run ./gopa -h to get the full list of command line options

  • -log option : logging level,can be set to trace,debug,info,warn,error ,default is info
  • -daemon option : run in background as daemon
  • -pprof option : enable and setup pprof/expvar service, eg: localhost:6060 , the endpoint will be: http://localhost:6060/debug/pprof/ and http://localhost:6060/debug/vars
  • -cpuprofile option : write cpu profile to this file
  • -memprofile option : write memory profile to this file

Stop

It's safety to press ctrl+c stop the current running Gopa, Gopa will handle the rest,saving the checkpoint, you may restore the job later,the world is still in your hand.

If you are running Gopa as daemon, you may stop it like this:

 kill -QUIT `pgrep gopa`

UI

Visit http://127.0.0.1:9001/ for more details.

License

Released under the Apache License, Version 2.0 .

About

GOPA, a spider written in Go. (QQ群号码:333088962)

Resources

License

Security policy

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 91.8%
  • HTML 6.9%
  • JavaScript 0.9%
  • Makefile 0.4%
  • Batchfile 0.0%
  • Shell 0.0%