mirror of
https://github.com/NohamR/papeer.git
synced 2026-05-25 20:00:47 +00:00
Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
108e1bdd1a | ||
|
|
be69854b17 | ||
|
|
d8a3cc027f | ||
|
|
be45a8f744 |
15
.github/workflows/format.yml
vendored
Normal file
15
.github/workflows/format.yml
vendored
Normal file
@@ -0,0 +1,15 @@
|
||||
name: Format
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- '**.go'
|
||||
|
||||
jobs:
|
||||
format:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v2
|
||||
- name: Check formatting
|
||||
run: if [[ -n "$(gofmt -l .)" ]]; then exit 1; fi
|
||||
15
.github/workflows/test.yml
vendored
Normal file
15
.github/workflows/test.yml
vendored
Normal file
@@ -0,0 +1,15 @@
|
||||
name: Test
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- '**.go'
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v2
|
||||
- name: Test
|
||||
run: make test
|
||||
31
README.md
31
README.md
@@ -1,23 +1,22 @@
|
||||
# Papeer
|
||||
|
||||
Papeer is a powerful an **ereader internet vacuum**. It can scrape any website, removing ads and keeping only the relevant content (formatted text and images). You can export the content to Markdown, EPUB or MOBI files.
|
||||
Papeer is a powerful **ereader internet vacuum**. It can scrape any website, removing ads and keeping only the relevant content (formatted text and images). You can export the content to Markdown, EPUB or MOBI files.
|
||||
|
||||
# Table of contents
|
||||
|
||||
- [Usage](#usage)
|
||||
- [Examples](#examples)
|
||||
* [Grab a single page](#grab-a-single-page)
|
||||
* [Grab several pages](#grab-several-pages)
|
||||
+ [`selector` option](#-selector--option)
|
||||
* [Scrape a web page](#scrape-a-web-page)
|
||||
* [Scrape a whole website](#scrape-a-whole-website)
|
||||
+ [`depth` option](#-depth--option)
|
||||
+ [Display table of contents](#display-table-of-contents)
|
||||
+ [`selector` option](#-selector--option)
|
||||
+ [Display the table of contents](#display-the-table-of-contents)
|
||||
+ [Scrape time](#scrape-time)
|
||||
- [Installation](#installation)
|
||||
* [From source](#from-source)
|
||||
* [From binary](#from-binary)
|
||||
+ [Linux / MacOS](#linux---macos)
|
||||
+ [Windows](#windows)
|
||||
* [MOBI support (optional)](#mobi-support--optional-)
|
||||
* [MOBI support](#mobi-support)
|
||||
- [Autocompletion](#autocompletion)
|
||||
- [Dependencies](#dependencies)
|
||||
|
||||
@@ -25,7 +24,7 @@ Papeer is a powerful an **ereader internet vacuum**. It can scrape any website,
|
||||
|
||||
## Scrape a web page
|
||||
|
||||
The `get` command lets you retrieve the content of any web page.
|
||||
The `get` command lets you retrieve the content of any web page or RSS feed.
|
||||
|
||||
```
|
||||
Scrape URL content
|
||||
@@ -49,6 +48,7 @@ Flags:
|
||||
-o, --offset int skip first chapters, use with depth/selector
|
||||
--output string file name (default: book name)
|
||||
-q, --quiet hide progress bar
|
||||
-r, --reverse reverse chapter order
|
||||
-s, --selector strings table of contents CSS selector
|
||||
-t, --threads int download concurrency, use with depth/selector (default -1)
|
||||
--use-link-name use link name for chapter title
|
||||
@@ -64,9 +64,9 @@ You can activate this mode by using the `depth` or `selector` options.
|
||||
|
||||
This option defaults to 0, `papeer` will grab only the main page.
|
||||
|
||||
If you specify a value greater than 0, `papeer` will grab only the pages as deep as the value you specify.
|
||||
If you specify a value greater than 0, `papeer` will grab pages as deep as the value you specify.
|
||||
|
||||
> Using `include` option will include all intermediary levels.
|
||||
> Using `include` option will include all intermediary levels into the book.
|
||||
|
||||
### `selector` option
|
||||
|
||||
@@ -76,9 +76,9 @@ If this option is specified, `papeer` will select the links (a HTML tag) present
|
||||
|
||||
You can chain this option to grab several level of pages with diferent selectors for each level.
|
||||
|
||||
## Display the table of contents
|
||||
### Display the table of contents
|
||||
|
||||
Before actually scraping a whole website, it is a good idea to use the `list` command. This command is like a **dry run**, which lets you vizualize the content before actually retrieving it. You can use several options to customize the table of contents extraction, such as `selector`, `limit`, `offset` and `include`. Type `papeer list --help` for more information about those options.
|
||||
Before actually scraping a whole website, it is a good idea to use the `list` command. This command is like a **dry run**, which lets you vizualize the content before actually retrieving it. You can use several options to customize the table of contents extraction, such as `selector`, `limit`, `offset`, `reverse` and `include`. Type `papeer list --help` for more information about those options.
|
||||
|
||||
```sh
|
||||
papeer list https://12factor.net/ -s 'section.concrete>article>h2>a'
|
||||
@@ -138,7 +138,7 @@ go get -u github.com/lapwat/papeer
|
||||
```sh
|
||||
# use platform=darwin for MacOS
|
||||
platform=linux
|
||||
release=0.4.0
|
||||
release=0.5.0
|
||||
|
||||
# download and extract
|
||||
curl -L https://github.com/lapwat/papeer/releases/download/v$release/papeer-v$release-$platform-amd64.tar.gz > papeer.tar.gz
|
||||
@@ -146,15 +146,14 @@ tar xzvf papeer.tar.gz
|
||||
rm papeer.tar.gz
|
||||
|
||||
# move to user binaries
|
||||
chmod +x papeer
|
||||
sudo mv papeer /usr/local/bin
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Download [latest release](https://github.com/lapwat/papeer/releases/download/v0.4.0/papeer-v0.4.0-windows-amd64.exe).
|
||||
Download [latest release](https://github.com/lapwat/papeer/releases/download/v0.5.0/papeer-v0.5.0-windows-amd64.exe.zip).
|
||||
|
||||
## (optional) MOBI support
|
||||
## MOBI support
|
||||
|
||||
Install kindlegen to convert websites, Linux only
|
||||
|
||||
|
||||
142
book/scraper.go
142
book/scraper.go
@@ -15,6 +15,7 @@ import (
|
||||
"github.com/PuerkitoBio/goquery"
|
||||
readability "github.com/go-shiori/go-readability"
|
||||
colly "github.com/gocolly/colly/v2"
|
||||
"github.com/mmcdole/gofeed"
|
||||
)
|
||||
|
||||
type ScrapeConfig struct {
|
||||
@@ -23,6 +24,7 @@ type ScrapeConfig struct {
|
||||
Quiet bool
|
||||
Limit int
|
||||
Offset int
|
||||
Reverse bool
|
||||
Delay int
|
||||
Threads int
|
||||
Include bool
|
||||
@@ -31,7 +33,7 @@ type ScrapeConfig struct {
|
||||
}
|
||||
|
||||
func NewScrapeConfig() *ScrapeConfig {
|
||||
return &ScrapeConfig{0, "", false, -1, 0, -1, -1, true, false, false}
|
||||
return &ScrapeConfig{0, "", false, -1, 0, false, -1, -1, true, false, false}
|
||||
}
|
||||
|
||||
func NewScrapeConfigs(selectors []string) []*ScrapeConfig {
|
||||
@@ -167,21 +169,24 @@ func NewChapterFromURL(url, linkName string, configs []*ScrapeConfig, index int,
|
||||
updateProgressBarName(index, name)
|
||||
}
|
||||
|
||||
subchapters := []chapter{}
|
||||
var subchapters []chapter
|
||||
if len(configs) > 1 {
|
||||
// add subchapters
|
||||
|
||||
links, _, _, err := GetLinks(base, config.Selector, config.Limit, config.Offset, false)
|
||||
// retrieve links on page
|
||||
links, _, _, err := GetLinks(base, config.Selector, config.Limit, config.Offset, config.Reverse, false)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
subchapters = make([]chapter, len(links))
|
||||
// init progess bar
|
||||
var p progress
|
||||
if config.Quiet == false {
|
||||
p = NewProgress(links, name, config.Depth)
|
||||
}
|
||||
|
||||
// init chapters list
|
||||
subchapters = make([]chapter, len(links))
|
||||
|
||||
if config.Delay >= 0 {
|
||||
|
||||
// synchronous mode
|
||||
@@ -277,7 +282,7 @@ func tableOfContent(url string, config *ScrapeConfig, subConfig *ScrapeConfig, q
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
links, _, home, err := GetLinks(base, config.Selector, config.Limit, config.Offset, config.Include)
|
||||
links, _, home, err := GetLinks(base, config.Selector, config.Limit, config.Offset, config.Reverse, config.Include)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
@@ -370,57 +375,76 @@ func GetPath(elm *goquery.Selection) string {
|
||||
return join
|
||||
}
|
||||
|
||||
func GetLinks(url *urllib.URL, selector string, limit, offset int, include bool) ([]link, string, chapter, error) {
|
||||
selectorSet := true
|
||||
if len(selector) == 0 {
|
||||
selector = "a"
|
||||
selectorSet = false
|
||||
func GetLinks(url *urllib.URL, selector string, limit, offset int, reverse, include bool) ([]link, string, chapter, error) {
|
||||
var links []link
|
||||
var pathMax string
|
||||
|
||||
parser := gofeed.NewParser()
|
||||
feed, err := parser.ParseURL(url.String())
|
||||
|
||||
if err == nil {
|
||||
// RSS feed
|
||||
|
||||
for _, item := range feed.Items {
|
||||
links = append(links, NewLink(item.Link, item.Title))
|
||||
}
|
||||
|
||||
pathMax = "RSS"
|
||||
} else {
|
||||
// HTML website
|
||||
|
||||
selectorSet := true
|
||||
if len(selector) == 0 {
|
||||
selector = "a"
|
||||
selectorSet = false
|
||||
}
|
||||
|
||||
pathLinks := map[string][]link{}
|
||||
pathCount := map[string]int{}
|
||||
pathMax = ""
|
||||
|
||||
// visit and count link classes
|
||||
c := colly.NewCollector()
|
||||
c.OnHTML(selector, func(e *colly.HTMLElement) {
|
||||
href := e.Attr("href")
|
||||
text := strings.TrimSpace(e.Text)
|
||||
path := GetPath(e.DOM)
|
||||
key := path
|
||||
|
||||
if selectorSet {
|
||||
|
||||
// if selector is set, we use the selector specified by the user
|
||||
|
||||
key = selector
|
||||
pathLinks[key] = append(pathLinks[key], NewLink(href, text))
|
||||
pathCount[key] += 1
|
||||
pathMax = key
|
||||
|
||||
} else {
|
||||
|
||||
// if selector is not set, we compute the selector ourselves
|
||||
|
||||
class := e.Attr("class")
|
||||
// include the element class to make sure we have the same exact path for every link in the table of content
|
||||
key = fmt.Sprintf("%s.%s", path, class)
|
||||
|
||||
// we count this key if the link text is not empty
|
||||
if text != "" {
|
||||
pathLinks[key] = append(pathLinks[key], NewLink(href, text))
|
||||
pathCount[key] += len(text)
|
||||
|
||||
if pathCount[key] > pathCount[pathMax] {
|
||||
pathMax = key
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
})
|
||||
c.Visit(url.String())
|
||||
|
||||
links = pathLinks[pathMax]
|
||||
}
|
||||
|
||||
pathLinks := map[string][]link{}
|
||||
pathCount := map[string]int{}
|
||||
pathMax := ""
|
||||
|
||||
// visit and count link classes
|
||||
c := colly.NewCollector()
|
||||
c.OnHTML(selector, func(e *colly.HTMLElement) {
|
||||
href := e.Attr("href")
|
||||
text := strings.TrimSpace(e.Text)
|
||||
path := GetPath(e.DOM)
|
||||
key := path
|
||||
|
||||
if selectorSet {
|
||||
|
||||
// if selector is set, we use the selector specified by the user
|
||||
|
||||
key = selector
|
||||
pathLinks[key] = append(pathLinks[key], NewLink(href, text))
|
||||
pathCount[key] += 1
|
||||
pathMax = key
|
||||
|
||||
} else {
|
||||
|
||||
// if selector is not set, we compute the selector ourselves
|
||||
|
||||
class := e.Attr("class")
|
||||
// include the element class to make sure we have the same exact path for every link in the table of content
|
||||
key = fmt.Sprintf("%s.%s", path, class)
|
||||
|
||||
// we count this key if the link text is not empty
|
||||
if text != "" {
|
||||
pathLinks[key] = append(pathLinks[key], NewLink(href, text))
|
||||
pathCount[key] += len(text)
|
||||
|
||||
if pathCount[key] > pathCount[pathMax] {
|
||||
pathMax = key
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
})
|
||||
c.Visit(url.String())
|
||||
|
||||
links := pathLinks[pathMax]
|
||||
if len(links) == 0 {
|
||||
return []link{}, pathMax, chapter{}, fmt.Errorf("no link found for selector: %s", selector)
|
||||
}
|
||||
@@ -434,10 +458,18 @@ func GetLinks(url *urllib.URL, selector string, limit, offset int, include bool)
|
||||
|
||||
home := NewChapterFromURL(url.String(), "", []*ScrapeConfig{NewScrapeConfig()}, 0, func(index int, name string) {})
|
||||
|
||||
// include home page
|
||||
if include {
|
||||
l := NewLink(url.String(), home.Name())
|
||||
links = append([]link{l}, links...)
|
||||
}
|
||||
|
||||
// reverse links
|
||||
if reverse {
|
||||
for i, j := 0, len(links)-1; i < j; i, j = i+1, j-1 {
|
||||
links[i], links[j] = links[j], links[i]
|
||||
}
|
||||
}
|
||||
|
||||
return links, pathMax, home, nil
|
||||
}
|
||||
|
||||
@@ -128,6 +128,22 @@ func TestSubChapters(t *testing.T) {
|
||||
|
||||
}
|
||||
|
||||
func TestSubChaptersRSS(t *testing.T) {
|
||||
|
||||
config0 := NewScrapeConfig()
|
||||
config1 := NewScrapeConfig()
|
||||
|
||||
c := NewChapterFromURL("https://blog.lapw.at/rss", "", []*ScrapeConfig{config0, config1}, 0, func(index int, name string) {})
|
||||
|
||||
got := len(c.SubChapters())
|
||||
want := 8
|
||||
|
||||
if got != want {
|
||||
t.Errorf("got %v, wanted %v", got, want)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
func TestSubChaptersSelector(t *testing.T) {
|
||||
|
||||
config0 := NewScrapeConfig()
|
||||
@@ -182,6 +198,24 @@ func TestSubChaptersLimitOver(t *testing.T) {
|
||||
|
||||
}
|
||||
|
||||
func TestReverse(t *testing.T) {
|
||||
|
||||
config0 := NewScrapeConfig()
|
||||
config0.Reverse = true
|
||||
|
||||
config1 := NewScrapeConfig()
|
||||
|
||||
c := NewChapterFromURL("https://books.lapw.at/", "", []*ScrapeConfig{config0, config1}, 0, func(index int, name string) {})
|
||||
|
||||
got := c.SubChapters()[0].Name()
|
||||
want := "The Twelve-Factor App"
|
||||
|
||||
if got != want {
|
||||
t.Errorf("got %v, wanted %v", got, want)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
func TestNotInclude(t *testing.T) {
|
||||
|
||||
config := NewScrapeConfig()
|
||||
|
||||
14
cmd/get.go
14
cmd/get.go
@@ -25,6 +25,7 @@ type GetOptions struct {
|
||||
depth int
|
||||
limit int
|
||||
offset int
|
||||
reverse bool
|
||||
delay int
|
||||
threads int
|
||||
// includeUrl bool
|
||||
@@ -49,6 +50,7 @@ func init() {
|
||||
getCmd.Flags().IntVarP(&getOpts.depth, "depth", "d", 0, "scraping depth")
|
||||
getCmd.Flags().IntVarP(&getOpts.limit, "limit", "l", -1, "limit number of chapters, use with depth/selector")
|
||||
getCmd.Flags().IntVarP(&getOpts.offset, "offset", "o", 0, "skip first chapters, use with depth/selector")
|
||||
getCmd.Flags().BoolVarP(&getOpts.reverse, "reverse", "r", false, "reverse chapter order")
|
||||
getCmd.Flags().IntVarP(&getOpts.delay, "delay", "", -1, "time in milliseconds to wait before downloading next chapter, use with depth/selector")
|
||||
getCmd.Flags().IntVarP(&getOpts.threads, "threads", "t", -1, "download concurrency, use with depth/selector")
|
||||
getCmd.Flags().BoolVarP(&getOpts.include, "include", "i", false, "include URL as first chapter, use with depth/selector")
|
||||
@@ -96,6 +98,10 @@ var getCmd = &cobra.Command{
|
||||
return errors.New("cannot use offset option if depth/selector is not specified")
|
||||
}
|
||||
|
||||
if cmd.Flags().Changed("reverse") && getOpts.depth == 0 && len(getOpts.Selector) == 0 {
|
||||
return errors.New("cannot use reverse option if depth/selector is not specified")
|
||||
}
|
||||
|
||||
if cmd.Flags().Changed("delay") && getOpts.depth == 0 && len(getOpts.Selector) == 0 {
|
||||
return errors.New("cannot use delay option if depth/selector is not specified")
|
||||
}
|
||||
@@ -104,6 +110,10 @@ var getCmd = &cobra.Command{
|
||||
return errors.New("cannot use threads option if depth/selector is not specified")
|
||||
}
|
||||
|
||||
if cmd.Flags().Changed("use-link-name") && getOpts.depth == 0 && len(getOpts.Selector) == 0 {
|
||||
return errors.New("cannot use use-link-name option if depth/selector is not specified")
|
||||
}
|
||||
|
||||
if cmd.Flags().Changed("delay") && cmd.Flags().Changed("threads") {
|
||||
return errors.New("cannot use delay and threads options at the same time")
|
||||
}
|
||||
@@ -114,7 +124,8 @@ var getCmd = &cobra.Command{
|
||||
url := args[0]
|
||||
|
||||
// fill selector array with empty selectors to match depth
|
||||
for len(getOpts.Selector) < getOpts.depth+2 {
|
||||
getOpts.Selector = append(getOpts.Selector, "")
|
||||
for len(getOpts.Selector) < getOpts.depth+1 {
|
||||
getOpts.Selector = append(getOpts.Selector, "")
|
||||
}
|
||||
|
||||
@@ -126,6 +137,7 @@ var getCmd = &cobra.Command{
|
||||
config.Quiet = getOpts.quiet
|
||||
config.Limit = getOpts.limit
|
||||
config.Offset = getOpts.offset
|
||||
config.Reverse = getOpts.reverse
|
||||
config.Delay = getOpts.delay
|
||||
config.Threads = getOpts.threads
|
||||
config.ImagesOnly = getOpts.images
|
||||
|
||||
@@ -21,6 +21,7 @@ type ListOptions struct {
|
||||
depth int
|
||||
limit int
|
||||
offset int
|
||||
reverse bool
|
||||
delay int
|
||||
threads int
|
||||
// includeUrl bool
|
||||
@@ -33,10 +34,12 @@ var listOpts *ListOptions
|
||||
func init() {
|
||||
listOpts = &ListOptions{}
|
||||
|
||||
// common with get command
|
||||
listCmd.Flags().StringSliceVarP(&listOpts.Selector, "selector", "s", []string{}, "table of contents CSS selector")
|
||||
listCmd.Flags().IntVarP(&listOpts.depth, "depth", "d", 0, "scraping depth")
|
||||
listCmd.Flags().IntVarP(&listOpts.limit, "limit", "l", -1, "limit number of chapters, use with depth/selector")
|
||||
listCmd.Flags().IntVarP(&listOpts.offset, "offset", "o", 0, "skip first chapters, use with depth/selector")
|
||||
listCmd.Flags().BoolVarP(&listOpts.reverse, "reverse", "r", false, "reverse chapter order")
|
||||
listCmd.Flags().IntVarP(&listOpts.delay, "delay", "", -1, "time in milliseconds to wait before downloading next chapter, use with depth/selector")
|
||||
listCmd.Flags().IntVarP(&listOpts.threads, "threads", "t", -1, "download concurrency, use with depth/selector")
|
||||
listCmd.Flags().BoolVarP(&listOpts.include, "include", "i", false, "include URL as first chapter, use with depth/selector")
|
||||
@@ -66,7 +69,7 @@ var listCmd = &cobra.Command{
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
links, path, _, err := book.GetLinks(base, listOpts.Selector[0], listOpts.limit, listOpts.offset, listOpts.include)
|
||||
links, path, _, err := book.GetLinks(base, listOpts.Selector[0], listOpts.limit, listOpts.offset, listOpts.reverse, listOpts.include)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -14,6 +14,6 @@ var versionCmd = &cobra.Command{
|
||||
Use: "version",
|
||||
Short: "Print the version number of papeer",
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
fmt.Println("papeer v0.4.0")
|
||||
fmt.Println("papeer v0.5.0")
|
||||
},
|
||||
}
|
||||
|
||||
5
go.mod
5
go.mod
@@ -29,10 +29,15 @@ require (
|
||||
github.com/golang/protobuf v1.5.2 // indirect
|
||||
github.com/inconshreveable/mousetrap v1.0.0 // indirect
|
||||
github.com/jedib0t/go-pretty/v6 v6.2.4 // indirect
|
||||
github.com/json-iterator/go v1.1.11 // indirect
|
||||
github.com/kennygrant/sanitize v1.2.4 // indirect
|
||||
github.com/mattn/go-isatty v0.0.14 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.13 // indirect
|
||||
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
|
||||
github.com/mmcdole/gofeed v1.1.3 // indirect
|
||||
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf // indirect
|
||||
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 // indirect
|
||||
github.com/modern-go/reflect2 v1.0.1 // indirect
|
||||
github.com/rivo/uniseg v0.2.0 // indirect
|
||||
github.com/saintfish/chardet v0.0.0-20120816061221-3af4cd4741ca // indirect
|
||||
github.com/schollz/progressbar/v3 v3.8.3 // indirect
|
||||
|
||||
10
go.sum
10
go.sum
@@ -87,6 +87,7 @@ github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3Ee
|
||||
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
|
||||
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
|
||||
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
@@ -235,6 +236,8 @@ github.com/jawher/mow.cli v1.1.0/go.mod h1:aNaQlc7ozF3vw6IJ2dHjp2ZFiA4ozMIYY6Pyu
|
||||
github.com/jedib0t/go-pretty/v6 v6.2.4 h1:wdaj2KHD2W+mz8JgJ/Q6L/T5dB7kyqEFI16eLq7GEmk=
|
||||
github.com/jedib0t/go-pretty/v6 v6.2.4/go.mod h1:+nE9fyyHGil+PuISTCrp7avEdo6bqoMwqZnuiK2r2a0=
|
||||
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
|
||||
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
|
||||
github.com/json-iterator/go v1.1.11 h1:uVUAXhF2To8cbw/3xN3pxj6kk7TYKs98NIrTqPlMWAQ=
|
||||
github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
|
||||
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
|
||||
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
|
||||
@@ -275,8 +278,14 @@ github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0Qu
|
||||
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
|
||||
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
|
||||
github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
|
||||
github.com/mmcdole/gofeed v1.1.3 h1:pdrvMb18jMSLidGp8j0pLvc9IGziX4vbmvVqmLH6z8o=
|
||||
github.com/mmcdole/gofeed v1.1.3/go.mod h1:QQO3maftbOu+hiVOGOZDRLymqGQCos4zxbA4j89gMrE=
|
||||
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf h1:sWGE2v+hO0Nd4yFU/S/mDBM5plIU8v/Qhfz41hkDIAI=
|
||||
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf/go.mod h1:pasqhqstspkosTneA62Nc+2p9SOBBYAPbnmRRWPQ0V8=
|
||||
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 h1:ZqeYNhU3OHLH3mGKHDcjJRFFRrJa6eAM5H+CtDdOsPc=
|
||||
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
|
||||
github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI=
|
||||
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
|
||||
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
|
||||
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
|
||||
@@ -355,6 +364,7 @@ github.com/temoto/robotstxt v1.1.2 h1:W2pOjSJ6SWvldyEuiFXNxz3xZ8aiWX5LbfDiOFd7Fx
|
||||
github.com/temoto/robotstxt v1.1.2/go.mod h1:+1AmkuG3IYkh1kv0d2qEB9Le88ehNO0zwOr3ujewlOo=
|
||||
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
|
||||
github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
|
||||
github.com/urfave/cli v1.22.3/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
|
||||
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
|
||||
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
|
||||
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
|
||||
26
release.sh
26
release.sh
@@ -1,34 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
version=$1
|
||||
platforms=("linux/amd64" "darwin/amd64" "windows/amd64")
|
||||
|
||||
if [ "$#" -ne 1 ]; then
|
||||
echo "Illegal number of parameters"
|
||||
echo "Usage: ./release.sh X.X.X"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
version=$1
|
||||
platforms=("linux/amd64" "darwin/amd64" "windows/amd64")
|
||||
|
||||
for platform in "${platforms[@]}"
|
||||
do
|
||||
platform_split=(${platform//\// })
|
||||
GOOS=${platform_split[0]}
|
||||
GOARCH=${platform_split[1]}
|
||||
output_name='papeer-v'$version'-'$GOOS'-'$GOARCH
|
||||
if [ $GOOS = "windows" ]; then
|
||||
output_name+='.exe'
|
||||
fi
|
||||
|
||||
env GOOS=$GOOS GOARCH=$GOARCH go build -o $output_name
|
||||
if [ $? -ne 0 ]; then
|
||||
echo 'An error has occurred! Aborting the script execution...'
|
||||
exit 1
|
||||
fi
|
||||
output_name=papeer
|
||||
|
||||
if [ $GOOS = "windows" ]; then
|
||||
zip "$output_name.exe.zip" "$output_name"
|
||||
env GOOS=$GOOS GOARCH=$GOARCH go build -o "$output_name.exe"
|
||||
zip "$output_name-v$version-$GOOS-$GOARCH.exe.zip" "$output_name.exe"
|
||||
rm "$output_name.exe"
|
||||
else
|
||||
tar czvf "$output_name.tar.gz" "$output_name"
|
||||
env GOOS=$GOOS GOARCH=$GOARCH go build -o "$output_name"
|
||||
tar czvf "$output_name-v$version-$GOOS-$GOARCH.tar.gz" "$output_name"
|
||||
rm "$output_name"
|
||||
fi
|
||||
rm "$output_name"
|
||||
done
|
||||
|
||||
Reference in New Issue
Block a user