4 Commits

Author SHA1 Message Date
lapwat
108e1bdd1a handle RSS feed, add CI 2022-03-02 19:06:55 +01:00
lapwat
be69854b17 add reverse option 2022-02-21 00:32:39 +01:00
lapwat
d8a3cc027f fix: selector depth 2022-02-06 23:35:35 +01:00
lapwat
be45a8f744 update installation instructions 2022-02-05 11:58:02 +01:00
11 changed files with 209 additions and 90 deletions

15
.github/workflows/format.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
name: Format
on:
push:
paths:
- '**.go'
jobs:
format:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Check formatting
run: if [[ -n "$(gofmt -l .)" ]]; then exit 1; fi

15
.github/workflows/test.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
name: Test
on:
push:
paths:
- '**.go'
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Test
run: make test

View File

@@ -1,23 +1,22 @@
# Papeer # Papeer
Papeer is a powerful an **ereader internet vacuum**. It can scrape any website, removing ads and keeping only the relevant content (formatted text and images). You can export the content to Markdown, EPUB or MOBI files. Papeer is a powerful **ereader internet vacuum**. It can scrape any website, removing ads and keeping only the relevant content (formatted text and images). You can export the content to Markdown, EPUB or MOBI files.
# Table of contents # Table of contents
- [Usage](#usage) - [Usage](#usage)
- [Examples](#examples) * [Scrape a web page](#scrape-a-web-page)
* [Grab a single page](#grab-a-single-page) * [Scrape a whole website](#scrape-a-whole-website)
* [Grab several pages](#grab-several-pages)
+ [`selector` option](#-selector--option)
+ [`depth` option](#-depth--option) + [`depth` option](#-depth--option)
+ [Display table of contents](#display-table-of-contents) + [`selector` option](#-selector--option)
+ [Display the table of contents](#display-the-table-of-contents)
+ [Scrape time](#scrape-time) + [Scrape time](#scrape-time)
- [Installation](#installation) - [Installation](#installation)
* [From source](#from-source) * [From source](#from-source)
* [From binary](#from-binary) * [From binary](#from-binary)
+ [Linux / MacOS](#linux---macos) + [Linux / MacOS](#linux---macos)
+ [Windows](#windows) + [Windows](#windows)
* [MOBI support (optional)](#mobi-support--optional-) * [MOBI support](#mobi-support)
- [Autocompletion](#autocompletion) - [Autocompletion](#autocompletion)
- [Dependencies](#dependencies) - [Dependencies](#dependencies)
@@ -25,7 +24,7 @@ Papeer is a powerful an **ereader internet vacuum**. It can scrape any website,
## Scrape a web page ## Scrape a web page
The `get` command lets you retrieve the content of any web page. The `get` command lets you retrieve the content of any web page or RSS feed.
``` ```
Scrape URL content Scrape URL content
@@ -49,6 +48,7 @@ Flags:
-o, --offset int skip first chapters, use with depth/selector -o, --offset int skip first chapters, use with depth/selector
--output string file name (default: book name) --output string file name (default: book name)
-q, --quiet hide progress bar -q, --quiet hide progress bar
-r, --reverse reverse chapter order
-s, --selector strings table of contents CSS selector -s, --selector strings table of contents CSS selector
-t, --threads int download concurrency, use with depth/selector (default -1) -t, --threads int download concurrency, use with depth/selector (default -1)
--use-link-name use link name for chapter title --use-link-name use link name for chapter title
@@ -64,9 +64,9 @@ You can activate this mode by using the `depth` or `selector` options.
This option defaults to 0, `papeer` will grab only the main page. This option defaults to 0, `papeer` will grab only the main page.
If you specify a value greater than 0, `papeer` will grab only the pages as deep as the value you specify. If you specify a value greater than 0, `papeer` will grab pages as deep as the value you specify.
> Using `include` option will include all intermediary levels. > Using `include` option will include all intermediary levels into the book.
### `selector` option ### `selector` option
@@ -76,9 +76,9 @@ If this option is specified, `papeer` will select the links (a HTML tag) present
You can chain this option to grab several level of pages with diferent selectors for each level. You can chain this option to grab several level of pages with diferent selectors for each level.
## Display the table of contents ### Display the table of contents
Before actually scraping a whole website, it is a good idea to use the `list` command. This command is like a **dry run**, which lets you vizualize the content before actually retrieving it. You can use several options to customize the table of contents extraction, such as `selector`, `limit`, `offset` and `include`. Type `papeer list --help` for more information about those options. Before actually scraping a whole website, it is a good idea to use the `list` command. This command is like a **dry run**, which lets you vizualize the content before actually retrieving it. You can use several options to customize the table of contents extraction, such as `selector`, `limit`, `offset`, `reverse` and `include`. Type `papeer list --help` for more information about those options.
```sh ```sh
papeer list https://12factor.net/ -s 'section.concrete>article>h2>a' papeer list https://12factor.net/ -s 'section.concrete>article>h2>a'
@@ -138,7 +138,7 @@ go get -u github.com/lapwat/papeer
```sh ```sh
# use platform=darwin for MacOS # use platform=darwin for MacOS
platform=linux platform=linux
release=0.4.0 release=0.5.0
# download and extract # download and extract
curl -L https://github.com/lapwat/papeer/releases/download/v$release/papeer-v$release-$platform-amd64.tar.gz > papeer.tar.gz curl -L https://github.com/lapwat/papeer/releases/download/v$release/papeer-v$release-$platform-amd64.tar.gz > papeer.tar.gz
@@ -146,15 +146,14 @@ tar xzvf papeer.tar.gz
rm papeer.tar.gz rm papeer.tar.gz
# move to user binaries # move to user binaries
chmod +x papeer
sudo mv papeer /usr/local/bin sudo mv papeer /usr/local/bin
``` ```
### Windows ### Windows
Download [latest release](https://github.com/lapwat/papeer/releases/download/v0.4.0/papeer-v0.4.0-windows-amd64.exe). Download [latest release](https://github.com/lapwat/papeer/releases/download/v0.5.0/papeer-v0.5.0-windows-amd64.exe.zip).
## (optional) MOBI support ## MOBI support
Install kindlegen to convert websites, Linux only Install kindlegen to convert websites, Linux only

View File

@@ -15,6 +15,7 @@ import (
"github.com/PuerkitoBio/goquery" "github.com/PuerkitoBio/goquery"
readability "github.com/go-shiori/go-readability" readability "github.com/go-shiori/go-readability"
colly "github.com/gocolly/colly/v2" colly "github.com/gocolly/colly/v2"
"github.com/mmcdole/gofeed"
) )
type ScrapeConfig struct { type ScrapeConfig struct {
@@ -23,6 +24,7 @@ type ScrapeConfig struct {
Quiet bool Quiet bool
Limit int Limit int
Offset int Offset int
Reverse bool
Delay int Delay int
Threads int Threads int
Include bool Include bool
@@ -31,7 +33,7 @@ type ScrapeConfig struct {
} }
func NewScrapeConfig() *ScrapeConfig { func NewScrapeConfig() *ScrapeConfig {
return &ScrapeConfig{0, "", false, -1, 0, -1, -1, true, false, false} return &ScrapeConfig{0, "", false, -1, 0, false, -1, -1, true, false, false}
} }
func NewScrapeConfigs(selectors []string) []*ScrapeConfig { func NewScrapeConfigs(selectors []string) []*ScrapeConfig {
@@ -167,21 +169,24 @@ func NewChapterFromURL(url, linkName string, configs []*ScrapeConfig, index int,
updateProgressBarName(index, name) updateProgressBarName(index, name)
} }
subchapters := []chapter{} var subchapters []chapter
if len(configs) > 1 { if len(configs) > 1 {
// add subchapters
links, _, _, err := GetLinks(base, config.Selector, config.Limit, config.Offset, false) // retrieve links on page
links, _, _, err := GetLinks(base, config.Selector, config.Limit, config.Offset, config.Reverse, false)
if err != nil { if err != nil {
log.Fatal(err) log.Fatal(err)
} }
subchapters = make([]chapter, len(links)) // init progess bar
var p progress var p progress
if config.Quiet == false { if config.Quiet == false {
p = NewProgress(links, name, config.Depth) p = NewProgress(links, name, config.Depth)
} }
// init chapters list
subchapters = make([]chapter, len(links))
if config.Delay >= 0 { if config.Delay >= 0 {
// synchronous mode // synchronous mode
@@ -277,7 +282,7 @@ func tableOfContent(url string, config *ScrapeConfig, subConfig *ScrapeConfig, q
log.Fatal(err) log.Fatal(err)
} }
links, _, home, err := GetLinks(base, config.Selector, config.Limit, config.Offset, config.Include) links, _, home, err := GetLinks(base, config.Selector, config.Limit, config.Offset, config.Reverse, config.Include)
if err != nil { if err != nil {
log.Fatal(err) log.Fatal(err)
} }
@@ -370,57 +375,76 @@ func GetPath(elm *goquery.Selection) string {
return join return join
} }
func GetLinks(url *urllib.URL, selector string, limit, offset int, include bool) ([]link, string, chapter, error) { func GetLinks(url *urllib.URL, selector string, limit, offset int, reverse, include bool) ([]link, string, chapter, error) {
selectorSet := true var links []link
if len(selector) == 0 { var pathMax string
selector = "a"
selectorSet = false parser := gofeed.NewParser()
feed, err := parser.ParseURL(url.String())
if err == nil {
// RSS feed
for _, item := range feed.Items {
links = append(links, NewLink(item.Link, item.Title))
}
pathMax = "RSS"
} else {
// HTML website
selectorSet := true
if len(selector) == 0 {
selector = "a"
selectorSet = false
}
pathLinks := map[string][]link{}
pathCount := map[string]int{}
pathMax = ""
// visit and count link classes
c := colly.NewCollector()
c.OnHTML(selector, func(e *colly.HTMLElement) {
href := e.Attr("href")
text := strings.TrimSpace(e.Text)
path := GetPath(e.DOM)
key := path
if selectorSet {
// if selector is set, we use the selector specified by the user
key = selector
pathLinks[key] = append(pathLinks[key], NewLink(href, text))
pathCount[key] += 1
pathMax = key
} else {
// if selector is not set, we compute the selector ourselves
class := e.Attr("class")
// include the element class to make sure we have the same exact path for every link in the table of content
key = fmt.Sprintf("%s.%s", path, class)
// we count this key if the link text is not empty
if text != "" {
pathLinks[key] = append(pathLinks[key], NewLink(href, text))
pathCount[key] += len(text)
if pathCount[key] > pathCount[pathMax] {
pathMax = key
}
}
}
})
c.Visit(url.String())
links = pathLinks[pathMax]
} }
pathLinks := map[string][]link{}
pathCount := map[string]int{}
pathMax := ""
// visit and count link classes
c := colly.NewCollector()
c.OnHTML(selector, func(e *colly.HTMLElement) {
href := e.Attr("href")
text := strings.TrimSpace(e.Text)
path := GetPath(e.DOM)
key := path
if selectorSet {
// if selector is set, we use the selector specified by the user
key = selector
pathLinks[key] = append(pathLinks[key], NewLink(href, text))
pathCount[key] += 1
pathMax = key
} else {
// if selector is not set, we compute the selector ourselves
class := e.Attr("class")
// include the element class to make sure we have the same exact path for every link in the table of content
key = fmt.Sprintf("%s.%s", path, class)
// we count this key if the link text is not empty
if text != "" {
pathLinks[key] = append(pathLinks[key], NewLink(href, text))
pathCount[key] += len(text)
if pathCount[key] > pathCount[pathMax] {
pathMax = key
}
}
}
})
c.Visit(url.String())
links := pathLinks[pathMax]
if len(links) == 0 { if len(links) == 0 {
return []link{}, pathMax, chapter{}, fmt.Errorf("no link found for selector: %s", selector) return []link{}, pathMax, chapter{}, fmt.Errorf("no link found for selector: %s", selector)
} }
@@ -434,10 +458,18 @@ func GetLinks(url *urllib.URL, selector string, limit, offset int, include bool)
home := NewChapterFromURL(url.String(), "", []*ScrapeConfig{NewScrapeConfig()}, 0, func(index int, name string) {}) home := NewChapterFromURL(url.String(), "", []*ScrapeConfig{NewScrapeConfig()}, 0, func(index int, name string) {})
// include home page
if include { if include {
l := NewLink(url.String(), home.Name()) l := NewLink(url.String(), home.Name())
links = append([]link{l}, links...) links = append([]link{l}, links...)
} }
// reverse links
if reverse {
for i, j := 0, len(links)-1; i < j; i, j = i+1, j-1 {
links[i], links[j] = links[j], links[i]
}
}
return links, pathMax, home, nil return links, pathMax, home, nil
} }

View File

@@ -128,6 +128,22 @@ func TestSubChapters(t *testing.T) {
} }
func TestSubChaptersRSS(t *testing.T) {
config0 := NewScrapeConfig()
config1 := NewScrapeConfig()
c := NewChapterFromURL("https://blog.lapw.at/rss", "", []*ScrapeConfig{config0, config1}, 0, func(index int, name string) {})
got := len(c.SubChapters())
want := 8
if got != want {
t.Errorf("got %v, wanted %v", got, want)
}
}
func TestSubChaptersSelector(t *testing.T) { func TestSubChaptersSelector(t *testing.T) {
config0 := NewScrapeConfig() config0 := NewScrapeConfig()
@@ -182,6 +198,24 @@ func TestSubChaptersLimitOver(t *testing.T) {
} }
func TestReverse(t *testing.T) {
config0 := NewScrapeConfig()
config0.Reverse = true
config1 := NewScrapeConfig()
c := NewChapterFromURL("https://books.lapw.at/", "", []*ScrapeConfig{config0, config1}, 0, func(index int, name string) {})
got := c.SubChapters()[0].Name()
want := "The Twelve-Factor App"
if got != want {
t.Errorf("got %v, wanted %v", got, want)
}
}
func TestNotInclude(t *testing.T) { func TestNotInclude(t *testing.T) {
config := NewScrapeConfig() config := NewScrapeConfig()

View File

@@ -25,6 +25,7 @@ type GetOptions struct {
depth int depth int
limit int limit int
offset int offset int
reverse bool
delay int delay int
threads int threads int
// includeUrl bool // includeUrl bool
@@ -49,6 +50,7 @@ func init() {
getCmd.Flags().IntVarP(&getOpts.depth, "depth", "d", 0, "scraping depth") getCmd.Flags().IntVarP(&getOpts.depth, "depth", "d", 0, "scraping depth")
getCmd.Flags().IntVarP(&getOpts.limit, "limit", "l", -1, "limit number of chapters, use with depth/selector") getCmd.Flags().IntVarP(&getOpts.limit, "limit", "l", -1, "limit number of chapters, use with depth/selector")
getCmd.Flags().IntVarP(&getOpts.offset, "offset", "o", 0, "skip first chapters, use with depth/selector") getCmd.Flags().IntVarP(&getOpts.offset, "offset", "o", 0, "skip first chapters, use with depth/selector")
getCmd.Flags().BoolVarP(&getOpts.reverse, "reverse", "r", false, "reverse chapter order")
getCmd.Flags().IntVarP(&getOpts.delay, "delay", "", -1, "time in milliseconds to wait before downloading next chapter, use with depth/selector") getCmd.Flags().IntVarP(&getOpts.delay, "delay", "", -1, "time in milliseconds to wait before downloading next chapter, use with depth/selector")
getCmd.Flags().IntVarP(&getOpts.threads, "threads", "t", -1, "download concurrency, use with depth/selector") getCmd.Flags().IntVarP(&getOpts.threads, "threads", "t", -1, "download concurrency, use with depth/selector")
getCmd.Flags().BoolVarP(&getOpts.include, "include", "i", false, "include URL as first chapter, use with depth/selector") getCmd.Flags().BoolVarP(&getOpts.include, "include", "i", false, "include URL as first chapter, use with depth/selector")
@@ -96,6 +98,10 @@ var getCmd = &cobra.Command{
return errors.New("cannot use offset option if depth/selector is not specified") return errors.New("cannot use offset option if depth/selector is not specified")
} }
if cmd.Flags().Changed("reverse") && getOpts.depth == 0 && len(getOpts.Selector) == 0 {
return errors.New("cannot use reverse option if depth/selector is not specified")
}
if cmd.Flags().Changed("delay") && getOpts.depth == 0 && len(getOpts.Selector) == 0 { if cmd.Flags().Changed("delay") && getOpts.depth == 0 && len(getOpts.Selector) == 0 {
return errors.New("cannot use delay option if depth/selector is not specified") return errors.New("cannot use delay option if depth/selector is not specified")
} }
@@ -104,6 +110,10 @@ var getCmd = &cobra.Command{
return errors.New("cannot use threads option if depth/selector is not specified") return errors.New("cannot use threads option if depth/selector is not specified")
} }
if cmd.Flags().Changed("use-link-name") && getOpts.depth == 0 && len(getOpts.Selector) == 0 {
return errors.New("cannot use use-link-name option if depth/selector is not specified")
}
if cmd.Flags().Changed("delay") && cmd.Flags().Changed("threads") { if cmd.Flags().Changed("delay") && cmd.Flags().Changed("threads") {
return errors.New("cannot use delay and threads options at the same time") return errors.New("cannot use delay and threads options at the same time")
} }
@@ -114,7 +124,8 @@ var getCmd = &cobra.Command{
url := args[0] url := args[0]
// fill selector array with empty selectors to match depth // fill selector array with empty selectors to match depth
for len(getOpts.Selector) < getOpts.depth+2 { getOpts.Selector = append(getOpts.Selector, "")
for len(getOpts.Selector) < getOpts.depth+1 {
getOpts.Selector = append(getOpts.Selector, "") getOpts.Selector = append(getOpts.Selector, "")
} }
@@ -126,6 +137,7 @@ var getCmd = &cobra.Command{
config.Quiet = getOpts.quiet config.Quiet = getOpts.quiet
config.Limit = getOpts.limit config.Limit = getOpts.limit
config.Offset = getOpts.offset config.Offset = getOpts.offset
config.Reverse = getOpts.reverse
config.Delay = getOpts.delay config.Delay = getOpts.delay
config.Threads = getOpts.threads config.Threads = getOpts.threads
config.ImagesOnly = getOpts.images config.ImagesOnly = getOpts.images

View File

@@ -21,6 +21,7 @@ type ListOptions struct {
depth int depth int
limit int limit int
offset int offset int
reverse bool
delay int delay int
threads int threads int
// includeUrl bool // includeUrl bool
@@ -33,10 +34,12 @@ var listOpts *ListOptions
func init() { func init() {
listOpts = &ListOptions{} listOpts = &ListOptions{}
// common with get command
listCmd.Flags().StringSliceVarP(&listOpts.Selector, "selector", "s", []string{}, "table of contents CSS selector") listCmd.Flags().StringSliceVarP(&listOpts.Selector, "selector", "s", []string{}, "table of contents CSS selector")
listCmd.Flags().IntVarP(&listOpts.depth, "depth", "d", 0, "scraping depth") listCmd.Flags().IntVarP(&listOpts.depth, "depth", "d", 0, "scraping depth")
listCmd.Flags().IntVarP(&listOpts.limit, "limit", "l", -1, "limit number of chapters, use with depth/selector") listCmd.Flags().IntVarP(&listOpts.limit, "limit", "l", -1, "limit number of chapters, use with depth/selector")
listCmd.Flags().IntVarP(&listOpts.offset, "offset", "o", 0, "skip first chapters, use with depth/selector") listCmd.Flags().IntVarP(&listOpts.offset, "offset", "o", 0, "skip first chapters, use with depth/selector")
listCmd.Flags().BoolVarP(&listOpts.reverse, "reverse", "r", false, "reverse chapter order")
listCmd.Flags().IntVarP(&listOpts.delay, "delay", "", -1, "time in milliseconds to wait before downloading next chapter, use with depth/selector") listCmd.Flags().IntVarP(&listOpts.delay, "delay", "", -1, "time in milliseconds to wait before downloading next chapter, use with depth/selector")
listCmd.Flags().IntVarP(&listOpts.threads, "threads", "t", -1, "download concurrency, use with depth/selector") listCmd.Flags().IntVarP(&listOpts.threads, "threads", "t", -1, "download concurrency, use with depth/selector")
listCmd.Flags().BoolVarP(&listOpts.include, "include", "i", false, "include URL as first chapter, use with depth/selector") listCmd.Flags().BoolVarP(&listOpts.include, "include", "i", false, "include URL as first chapter, use with depth/selector")
@@ -66,7 +69,7 @@ var listCmd = &cobra.Command{
log.Fatal(err) log.Fatal(err)
} }
links, path, _, err := book.GetLinks(base, listOpts.Selector[0], listOpts.limit, listOpts.offset, listOpts.include) links, path, _, err := book.GetLinks(base, listOpts.Selector[0], listOpts.limit, listOpts.offset, listOpts.reverse, listOpts.include)
if err != nil { if err != nil {
log.Fatal(err) log.Fatal(err)
} }

View File

@@ -14,6 +14,6 @@ var versionCmd = &cobra.Command{
Use: "version", Use: "version",
Short: "Print the version number of papeer", Short: "Print the version number of papeer",
Run: func(cmd *cobra.Command, args []string) { Run: func(cmd *cobra.Command, args []string) {
fmt.Println("papeer v0.4.0") fmt.Println("papeer v0.5.0")
}, },
} }

5
go.mod
View File

@@ -29,10 +29,15 @@ require (
github.com/golang/protobuf v1.5.2 // indirect github.com/golang/protobuf v1.5.2 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/jedib0t/go-pretty/v6 v6.2.4 // indirect github.com/jedib0t/go-pretty/v6 v6.2.4 // indirect
github.com/json-iterator/go v1.1.11 // indirect
github.com/kennygrant/sanitize v1.2.4 // indirect github.com/kennygrant/sanitize v1.2.4 // indirect
github.com/mattn/go-isatty v0.0.14 // indirect github.com/mattn/go-isatty v0.0.14 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
github.com/mmcdole/gofeed v1.1.3 // indirect
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf // indirect
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 // indirect
github.com/modern-go/reflect2 v1.0.1 // indirect
github.com/rivo/uniseg v0.2.0 // indirect github.com/rivo/uniseg v0.2.0 // indirect
github.com/saintfish/chardet v0.0.0-20120816061221-3af4cd4741ca // indirect github.com/saintfish/chardet v0.0.0-20120816061221-3af4cd4741ca // indirect
github.com/schollz/progressbar/v3 v3.8.3 // indirect github.com/schollz/progressbar/v3 v3.8.3 // indirect

10
go.sum
View File

@@ -87,6 +87,7 @@ github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3Ee
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc= github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
@@ -235,6 +236,8 @@ github.com/jawher/mow.cli v1.1.0/go.mod h1:aNaQlc7ozF3vw6IJ2dHjp2ZFiA4ozMIYY6Pyu
github.com/jedib0t/go-pretty/v6 v6.2.4 h1:wdaj2KHD2W+mz8JgJ/Q6L/T5dB7kyqEFI16eLq7GEmk= github.com/jedib0t/go-pretty/v6 v6.2.4 h1:wdaj2KHD2W+mz8JgJ/Q6L/T5dB7kyqEFI16eLq7GEmk=
github.com/jedib0t/go-pretty/v6 v6.2.4/go.mod h1:+nE9fyyHGil+PuISTCrp7avEdo6bqoMwqZnuiK2r2a0= github.com/jedib0t/go-pretty/v6 v6.2.4/go.mod h1:+nE9fyyHGil+PuISTCrp7avEdo6bqoMwqZnuiK2r2a0=
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.11 h1:uVUAXhF2To8cbw/3xN3pxj6kk7TYKs98NIrTqPlMWAQ=
github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
@@ -275,8 +278,14 @@ github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0Qu
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/mmcdole/gofeed v1.1.3 h1:pdrvMb18jMSLidGp8j0pLvc9IGziX4vbmvVqmLH6z8o=
github.com/mmcdole/gofeed v1.1.3/go.mod h1:QQO3maftbOu+hiVOGOZDRLymqGQCos4zxbA4j89gMrE=
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf h1:sWGE2v+hO0Nd4yFU/S/mDBM5plIU8v/Qhfz41hkDIAI=
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf/go.mod h1:pasqhqstspkosTneA62Nc+2p9SOBBYAPbnmRRWPQ0V8=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 h1:ZqeYNhU3OHLH3mGKHDcjJRFFRrJa6eAM5H+CtDdOsPc=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
@@ -355,6 +364,7 @@ github.com/temoto/robotstxt v1.1.2 h1:W2pOjSJ6SWvldyEuiFXNxz3xZ8aiWX5LbfDiOFd7Fx
github.com/temoto/robotstxt v1.1.2/go.mod h1:+1AmkuG3IYkh1kv0d2qEB9Le88ehNO0zwOr3ujewlOo= github.com/temoto/robotstxt v1.1.2/go.mod h1:+1AmkuG3IYkh1kv0d2qEB9Le88ehNO0zwOr3ujewlOo=
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc= github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
github.com/urfave/cli v1.22.3/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q= github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=

View File

@@ -1,34 +1,28 @@
#!/usr/bin/env bash #!/usr/bin/env bash
version=$1
platforms=("linux/amd64" "darwin/amd64" "windows/amd64")
if [ "$#" -ne 1 ]; then if [ "$#" -ne 1 ]; then
echo "Illegal number of parameters" echo "Illegal number of parameters"
echo "Usage: ./release.sh X.X.X" echo "Usage: ./release.sh X.X.X"
exit 1 exit 1
fi fi
version=$1
platforms=("linux/amd64" "darwin/amd64" "windows/amd64")
for platform in "${platforms[@]}" for platform in "${platforms[@]}"
do do
platform_split=(${platform//\// }) platform_split=(${platform//\// })
GOOS=${platform_split[0]} GOOS=${platform_split[0]}
GOARCH=${platform_split[1]} GOARCH=${platform_split[1]}
output_name='papeer-v'$version'-'$GOOS'-'$GOARCH output_name=papeer
if [ $GOOS = "windows" ]; then
output_name+='.exe'
fi
env GOOS=$GOOS GOARCH=$GOARCH go build -o $output_name
if [ $? -ne 0 ]; then
echo 'An error has occurred! Aborting the script execution...'
exit 1
fi
if [ $GOOS = "windows" ]; then if [ $GOOS = "windows" ]; then
zip "$output_name.exe.zip" "$output_name" env GOOS=$GOOS GOARCH=$GOARCH go build -o "$output_name.exe"
zip "$output_name-v$version-$GOOS-$GOARCH.exe.zip" "$output_name.exe"
rm "$output_name.exe"
else else
tar czvf "$output_name.tar.gz" "$output_name" env GOOS=$GOOS GOARCH=$GOARCH go build -o "$output_name"
tar czvf "$output_name-v$version-$GOOS-$GOARCH.tar.gz" "$output_name"
rm "$output_name"
fi fi
rm "$output_name"
done done