Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feed by tag #264

Merged
merged 31 commits into from Jun 22, 2020
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c2d2f3d
create a feed for each tag, so far it's empty, but baby steps
pmb00cs Feb 14, 2019
a5d5881
include logic to have inclusions or exclusions, and override default …
pmb00cs Feb 16, 2019
0bb481c
works to build tag feeds but generates warnings that need tracking down
pmb00cs Feb 17, 2019
223343b
minor style tweaks and add instructions to README
pmb00cs Feb 17, 2019
d2d31d7
remove rubocop exclusions
pmb00cs Feb 17, 2019
89eb58a
use rubocop to fix some styling errors, and correct it's mistakes
pmb00cs Feb 17, 2019
7601bc6
set some tags and use these to make tests for tag feeds
pmb00cs Feb 18, 2019
ee59d9e
refactor to reduce rubocop errors
pmb00cs Feb 18, 2019
c60d01e
change match to equivilency check to apease rubocop
pmb00cs Feb 18, 2019
f74c32f
move stuff out of logice block it doesn't need to live in
pmb00cs Feb 18, 2019
e622a6f
add tests for not setting tags so not building those feeds
pmb00cs Feb 18, 2019
8db0621
rubocop auto correct
pmb00cs Feb 18, 2019
6a7bd53
remove extra . that cause liquid syntax warnings
pmb00cs Feb 18, 2019
5f7d27a
refactor to simplify logic
ashmaroli Feb 19, 2019
665cfba
reset tags_config to empty Hash if it isn't Hash
ashmaroli Feb 19, 2019
67d817d
split method apart to reduce CyclomaticComplexity
ashmaroli Feb 19, 2019
f975bb5
Improve comment inlined within iteration logic
ashmaroli Feb 19, 2019
3eb88ac
fix comments from project maintainer
pmb00cs Feb 19, 2019
05ee178
Remove extra blank line in spec file
ashmaroli Feb 20, 2019
76bbf81
Merge branch 'master' of github.com:jekyll/jekyll-feed into feed_by_tag
pmb00cs Apr 6, 2019
d36c93a
Merge branch 'master' of github.com:jekyll/jekyll-feed into feed_by_tag
pmb00cs Nov 21, 2019
3d8e51e
change tag to tags in generator make_page function for consistency
pmb00cs Jan 10, 2020
9216296
add test for creating tag feed for tag created as a non-array tag in …
pmb00cs Jan 11, 2020
93c69a8
Test if questionable paths are handled properly
ashmaroli Jan 15, 2020
9cd44b4
Read `config["tags"]` just once
ashmaroli Jan 15, 2020
281b49f
change the names of settings for included or excluded feeds to make i…
pmb00cs Jan 15, 2020
fde5155
Line up = in generate_feed_by_tag function to make code more readable
pmb00cs Jan 15, 2020
36f4ea8
Merge branch 'master' of github.com:jekyll/jekyll-feed into feed_by_tag
pmb00cs May 10, 2020
2b21c5d
Merge branch 'master' of github.com:jekyll/jekyll-feed into feed_by_tag
pmb00cs May 15, 2020
27f8932
Merge branch 'master' of github.com:jekyll/jekyll-feed into feed_by_tag
pmb00cs Jun 22, 2020
d982285
remove test for core jekyll functionality suggested by parkr
pmb00cs Jun 22, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
38 changes: 38 additions & 0 deletions README.md
Expand Up @@ -196,6 +196,44 @@ feed:
The same flag can be used directly in post file. It will be disable `<content>` tag for selected post.
Settings in post file has higher priority than in config file.

## Tags

To automatically generate feeds for each tag you apply to your posts you can add a tags setting to your config:

```yml
feed:
tags: true
```
ashmaroli marked this conversation as resolved.
Show resolved Hide resolved

If there are tags you don't want included in this auto generation you can exclude them

```yml
feed:
tags:
except:
- tag-to-exclude
- another-tag
```

If you wish to change the location of these auto generated feeds (`/feed/by_tag/<TAG>.xml` by default) you can provide an alternative folder for them to live in.

```yml
feed:
tags:
path: "alternative/path/for/tags/feeds/"
```

If you only want to generate feeds for a few tags you can also set this.

```yml
feed:
tags:
only:
- tag-to-include
- another-tag
```

Note that if you include a tag that is excluded a feed will not be generated for it.

## Contributing

Expand Down
7 changes: 6 additions & 1 deletion lib/jekyll-feed/feed.xml
Expand Up @@ -39,7 +39,12 @@
</author>
{% endif %}

{% assign posts = site[page.collection] | where_exp: "post", "post.draft != true" | sort: "date" | reverse %}
{% if page.tags %}
{% assign entries = site.tags[page.tags] %}
{% else %}
{% assign entries = site[page.collection] %}
{% endif %}
{% assign posts = entries | where_exp: "post", "post.draft != true" | sort: "date" | reverse %}
{% if page.category %}
{% assign posts = posts | where: "category",page.category %}
{% endif %}
Expand Down
32 changes: 30 additions & 2 deletions lib/jekyll-feed/generator.rb
Expand Up @@ -17,6 +17,7 @@ def generate(site)
@site.pages << make_page(path, :collection => name, :category => category)
end
end
generate_feed_by_tag if config["tags"] && !@site.tags.empty?
end

private
Expand Down Expand Up @@ -69,6 +70,32 @@ def collections
@collections
end

def generate_feed_by_tag
tags_config = config["tags"]
tags_config = {} unless tags_config.is_a?(Hash)

except = tags_config["except"] || []
only = tags_config["only"] || @site.tags.keys
tags_pool = only - except
tags_path = tags_config["path"] || "/feed/by_tag/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if I pass in a malicious path here via the site's config? (e.g., ../../../../../some-secret-file)?

  • I think file_exists? below could reveal the file's existence?
  • It looks like we may be sanitizing the path to be within the plugin source? Is that possible or am I misreading?
  • If that's the case, then a malicious user may be able to write to an arbitrary path (potentially within the plugin source?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It gets passed to the same "make_page" function as generating the main feed uses. Honestly I'm not sure what happens with malicious paths get used, I assumed that function already included sanity checking for the path. The only change I made to that function was to allow it to include the option to pass in the tag you want a feed for.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benbalter In reality the @base and @path values never get used:

# jekyll/lib/jekyll/page.rb

def path
  data.fetch("path") { relative_path }
end

# The path to the page source file, relative to the site source
def relative_path
  @relative_path ||= File.join(*[@dir, @name].map(&:to_s).reject(&:empty?)).sub(%r!\A\/!, "")
end

@base which equals to the value of __dir__ is not used in our use-case because the :read_yaml is overridden:

def read_yaml(*)
@data ||= {}
end

That said, your observation cannot be left disregarded (also present in the master and published versions) since the instance variable could still be read via another third-party plugin when used outside the GitHub Pages environment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if I pass in a malicious path here via the site's config? (e.g., ../../../../../some-secret-file)?

Added a test for this in 93c69a8


generate_tag_feed(tags_pool, tags_path)
end

def generate_tag_feed(tags_pool, tags_path)
tags_pool.each do |tag|
# allow only tags with basic alphanumeric characters and underscore to keep
# feed path simple.
next if tag =~ %r![^a-zA-Z0-9_]!

Jekyll.logger.info "Jekyll Feed:", "Generating feed for posts tagged #{tag}"
path = "#{tags_path}#{tag}.xml"
next if file_exists?(path)

@site.pages << make_page(path, :tags => tag)
end
end

# Path to feed.xml template file
def feed_source_path
@feed_source_path ||= File.expand_path "feed.xml", __dir__
Expand All @@ -85,15 +112,16 @@ def file_exists?(file_path)

# Generates contents for a file

def make_page(file_path, collection: "posts", category: nil)
def make_page(file_path, collection: "posts", category: nil, tags: nil)
PageWithoutAFile.new(@site, __dir__, "", file_path).tap do |file|
file.content = feed_template
file.data.merge!(
"layout" => nil,
"sitemap" => false,
"xsl" => file_exists?("feed.xslt.xml"),
"collection" => collection,
"category" => category
"category" => category,
"tags" => tags
)
file.output
end
Expand Down
2 changes: 2 additions & 0 deletions spec/fixtures/_posts/2013-12-12-dec-the-second.md
Expand Up @@ -2,6 +2,8 @@
excerpt: "Foo"
image: "/image.png"
category: news
tags:
- test
---

# December the twelfth, actually.
2 changes: 2 additions & 0 deletions spec/fixtures/_posts/2014-03-02-march-the-second.md
@@ -1,6 +1,8 @@
---
image: https://cdn.example.org/absolute.png?h=188&w=250
category: news
tags:
- test
---

March the second!
1 change: 1 addition & 0 deletions spec/fixtures/_posts/2014-03-04-march-the-fourth.md
Expand Up @@ -2,6 +2,7 @@
title: <span class="highlight">Sparkling</span> Title
tags:
- '"/><VADER>'
- test
image:
path: "/object-image.png"
categories: updates jekyll
Expand Down
3 changes: 3 additions & 0 deletions spec/fixtures/_posts/2015-01-18-jekyll-last-modified-at.md
@@ -1,5 +1,8 @@
---
last_modified_at: 2015-05-12T13:27:59+00:00
tags:
- test
- fail
---

Please don't modify this file. It's modified time is important.
1 change: 1 addition & 0 deletions spec/fixtures/_posts/2015-05-12-liquid.md
@@ -1,4 +1,5 @@
---
tag: nonarray
---

{% capture liquidstring %}
Expand Down
2 changes: 2 additions & 0 deletions spec/fixtures/_posts/2015-05-12-pre.html
@@ -1,6 +1,8 @@
---
author: Pat
lang: en
tags:
- test
---

<pre>Line 1
Expand Down
3 changes: 3 additions & 0 deletions spec/fixtures/_posts/2015-05-18-author-detail.md
Expand Up @@ -4,6 +4,9 @@ author:
name: Ben
uri: "http://ben.balter.com"
email: ben@example.com
tags:
- test
- success
---

# December the twelfth, actually.
3 changes: 3 additions & 0 deletions spec/fixtures/_posts/2015-08-08-stuck-in-the-middle.html
@@ -1,4 +1,7 @@
---
tags:
- test
- fail
feed:
excerpt_only: true
---
Expand Down
154 changes: 154 additions & 0 deletions spec/jekyll-feed_spec.rb
Expand Up @@ -497,6 +497,160 @@
end
end

context "tags" do
let(:tags_feed_test) { File.read(dest_dir("feed/by_tag/test.xml")) }
let(:tags_feed_fail) { File.read(dest_dir("feed/by_tag/fail.xml")) }
let(:tags_feed_success) { File.read(dest_dir("feed/by_tag/success.xml")) }
let(:tags_feed_nonarray) { File.read(dest_dir("feed/by_tag/nonarray.xml")) }


context "do not set tags setting" do
it "should not write tags feeds" do
expect(Pathname.new(dest_dir("feed/by_tag/test.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/fail.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/success.xml"))).to_not exist
end
end

context "set tags setting" do
let(:overrides) do
{
"feed" => {
"tags" => true
},
}
end

it "should write tags feeds" do
expect(Pathname.new(dest_dir("feed/by_tag/test.xml"))).to exist
expect(Pathname.new(dest_dir("feed/by_tag/fail.xml"))).to exist
expect(Pathname.new(dest_dir("feed/by_tag/success.xml"))).to exist

expect(tags_feed_test).to match "/2013/12/12/dec-the-second.html"
expect(tags_feed_test).to match "/2014/03/02/march-the-second.html"
expect(tags_feed_test).to match "/2014/03/04/march-the-fourth.html"
expect(tags_feed_test).to match "/2015/01/18/jekyll-last-modified-at.html"
expect(tags_feed_test).to match "/2015/05/12/pre.html"
expect(tags_feed_test).to match "/2015/05/18/author-detail.html"
expect(tags_feed_test).to match "/2015/08/08/stuck-in-the-middle.html"

expect(tags_feed_fail).to match "/2015/01/18/jekyll-last-modified-at.html"
expect(tags_feed_fail).to match "/2015/08/08/stuck-in-the-middle.html"
expect(tags_feed_fail).to_not match "/2013/12/12/dec-the-second.html"
expect(tags_feed_fail).to_not match "/2014/03/02/march-the-second.html"
expect(tags_feed_fail).to_not match "/2014/03/04/march-the-fourth.html"
expect(tags_feed_fail).to_not match "/2015/05/12/pre.html"
expect(tags_feed_fail).to_not match "/2015/05/18/author-detail.html"

expect(tags_feed_success).to match "2015/05/18/author-detail.html"
expect(tags_feed_success).to_not match "/2013/12/12/dec-the-second.html"
expect(tags_feed_success).to_not match "/2014/03/02/march-the-second.html"
expect(tags_feed_success).to_not match "/2014/03/04/march-the-fourth.html"
expect(tags_feed_success).to_not match "/2015/01/18/jekyll-last-modified-at.html"
expect(tags_feed_success).to_not match "/2015/05/12/pre.html"
expect(tags_feed_success).to_not match "/2015/08/08/stuck-in-the-middle.html"
end
end

context "set exclusions" do
let(:overrides) do
{
"feed" => {
"tags" => {
"except" => ["fail"]
},
},
}
end

it "should not write fail feed" do
expect(Pathname.new(dest_dir("feed/by_tag/test.xml"))).to exist
expect(Pathname.new(dest_dir("feed/by_tag/fail.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/success.xml"))).to exist
end
end

context "set inclusions" do
let(:overrides) do
{
"feed" => {
"tags" => {
"only" => ["success"]
},
},
}
end

it "should not write fail feed" do
expect(Pathname.new(dest_dir("feed/by_tag/test.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/fail.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/success.xml"))).to exist
end
end

context "set alternate path" do
let(:overrides) do
{
"feed" => {
"tags" => {
"path" => "alternate/path/"
},
},
}
end

it "should write feeds to new path" do
expect(Pathname.new(dest_dir("feed/by_tag/test.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/fail.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/success.xml"))).to_not exist

expect(Pathname.new(dest_dir("alternate/path/test.xml"))).to exist
expect(Pathname.new(dest_dir("alternate/path/fail.xml"))).to exist
expect(Pathname.new(dest_dir("alternate/path/success.xml"))).to exist
end

context "set to questionable path" do
let(:overrides) do
{
"feed" => {
"tags" => {
"path" => "../../../../../../../questionable/path/"
},
},
}
end

it "should write feeds to sane paths" do
expect(Pathname.new(dest_dir("feed/by_tag/test.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/fail.xml"))).to_not exist
expect(Pathname.new(dest_dir("feed/by_tag/success.xml"))).to_not exist

expect(Pathname.new(dest_dir("questionable/path/test.xml"))).to exist
expect(Pathname.new(dest_dir("questionable/path/fail.xml"))).to exist
expect(Pathname.new(dest_dir("questionable/path/success.xml"))).to exist
end
end
end

context "test nonarray tag value" do
DirtyF marked this conversation as resolved.
Show resolved Hide resolved
let(:overrides) do
{
"feed" => {
"tags" => {
"only" => ["nonarray"]
},
},
}
end

it "should write nonarray feed" do
expect(Pathname.new(dest_dir("feed/by_tag/nonarray.xml"))).to exist
expect(tags_feed_nonarray).to match "/2015/05/12/liquid.html"
end

end
end

context "excerpt_only flag" do
context "backward compatibility for no excerpt_only flag" do
it "should be in contents" do
Expand Down