Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release - 0.4.0 #53

Merged
merged 131 commits into from Jul 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
441cd24
[#28] Enable paranoid mode to prevent account enumeration attacks
malparty Jun 23, 2021
6d682b3
[#29] Add style to reset password view
malparty Jun 23, 2021
c7584ac
[#29] Improve code quality of previous system tests for authentication
malparty Jun 24, 2021
4d723bf
[#29] Add reset password system tests
malparty Jun 24, 2021
73f7f33
[#29] Fix spec comments
malparty Jun 25, 2021
be83e11
Merge pull request #44 from malparty/feature/reset-password-backend
malparty Jun 28, 2021
f020025
Merge branch 'develop' into feature/reset-password-frontend
malparty Jun 28, 2021
e6f0fb8
Merge pull request #45 from malparty/feature/reset-password-frontend
malparty Jun 28, 2021
06eaae1
[#26] Add default keyword model
malparty Jun 24, 2021
560bc5e
[#26] Add keyword seeds for development ENV
malparty Jun 24, 2021
6949f18
[#26] Improve seeds using Fabricate.times
malparty Jun 24, 2021
4dbadbe
[#26] Add keyword query
malparty Jun 24, 2021
c8d8e84
[#26] Fix usage of pagy method
malparty Jun 24, 2021
7999c1c
[#26] Add keywords method in Keyword_controller to wrap Query Object
malparty Jun 28, 2021
2c4a4ca
[#26] Add keywords method in Keyword_controller to wrap Query Object
malparty Jun 28, 2021
7901109
[#26] Fix variable name conflict issue
malparty Jun 28, 2021
d2589a3
[#26] Add keyword validation and tests
malparty Jun 28, 2021
1f35b92
[#26] Rename tests descriptions to be easier
malparty Jun 28, 2021
c29ddf0
[#26] Rename tests descriptions to be easier
malparty Jun 28, 2021
53ba973
[#26] Rename order to oder_by_name
malparty Jun 28, 2021
af84dde
[#26] Add presence of name validation test for keyword model
malparty Jun 28, 2021
bd201a5
[#26] Use inline foreign key in DB migration
malparty Jun 28, 2021
a115206
[#26] Add soft-delete for users
malparty Jun 28, 2021
52348d9
[#26] Use shoulda-matchers test helpers for keyword validation tests
malparty Jun 28, 2021
f38d58c
[#26] Improve seed user declaration
malparty Jun 28, 2021
4deb0ec
[#26] Fix route using symbols
malparty Jun 28, 2021
926b251
[#26] Remove useless assignation in Keyword query
malparty Jun 28, 2021
f270465
[#26] Remove unused attr_reader in keywords query object
malparty Jun 29, 2021
24656e4
[#26] Add soft delete to keywords model
malparty Jun 29, 2021
bd24ff5
[#26] Use accessor to read private instance variable
malparty Jun 29, 2021
7e6a082
[#26] Add comment on overriden action registration#destroy
malparty Jun 29, 2021
7cbce3b
[#26] Reset simple quote in db/schema
malparty Jun 29, 2021
5a3282a
Merge pull request #46 from malparty/feature/list-keywords-backend
malparty Jun 30, 2021
074f17a
[#6] Add basic create action and form to perform a google query
malparty Jun 14, 2021
96843ba
[#5] Add lastname and firstname to registration views
malparty Jun 9, 2021
5a0848a
Rebase from develop pull
malparty Jun 16, 2021
6372b09
root commit
malparty Jun 11, 2021
604d04b
[#5] Rename lastname/firstname into last_name first_name
malparty Jun 14, 2021
9477eab
[#5] Merge email (citext) and renaming names into previous db migrations
malparty Jun 14, 2021
82c30ab
[#6] Handle error while query google - not colorized
malparty Jun 16, 2021
37c6c63
root commit
malparty Jun 11, 2021
520bd20
Rebase to develop - pull
malparty Jun 17, 2021
4e1ea37
root commit
malparty Jun 11, 2021
b317fd5
[#7] Instal nokogiri Html Parser
malparty Jun 15, 2021
e3a3539
[#7] Define parserservice interface
malparty Jun 15, 2021
21ed3d4
[#7] Implement ads_top_count
malparty Jun 15, 2021
8a350b3
[#7] Implement ads_page_count
malparty Jun 15, 2021
25be192
[#7] Implement ads_top_url
malparty Jun 15, 2021
31b709a
[#7] Add all other parsing methods
malparty Jun 16, 2021
844e5fe
root commit
malparty Jun 11, 2021
d1f1b05
[#5] Rename lastname/firstname into last_name first_name
malparty Jun 14, 2021
42d55fe
[#5] Merge email (citext) and renaming names into previous db migrations
malparty Jun 14, 2021
c397bd3
[#6] Declare vcr inline to improve code readability
malparty Jun 16, 2021
1f697b5
[#6] Replace instance variables by local variables inside view
malparty Jun 16, 2021
045ff30
[#6] Colorize logs from Rails.logger
malparty Jun 16, 2021
b6610c5
root commit
malparty Jun 11, 2021
2ae9256
root commit
malparty Jun 11, 2021
d35b731
root commit
malparty Jun 11, 2021
1a57cda
[#5] Update email db type to citext
malparty Jun 14, 2021
354e7da
[#5] Rename lastname/firstname into last_name first_name
malparty Jun 14, 2021
d30e668
[#5] Merge email (citext) and renaming names into previous db migrations
malparty Jun 14, 2021
9be2612
[#7] Fix rubocop warnings
malparty Jun 17, 2021
84d4330
[#7] Refactor tests with inline vcr
malparty Jun 17, 2021
b5abfeb
[#7] Add error handling into google parser
malparty Jun 18, 2021
12dd01f
root commit
malparty Jun 11, 2021
5248a3d
root commit
malparty Jun 11, 2021
e18701d
root commit
malparty Jun 11, 2021
954dab6
root commit
malparty Jun 11, 2021
27b18c2
[#6] Remove create action and index form
malparty Jun 21, 2021
ec6623e
root commit
malparty Jun 11, 2021
5f73d93
root commit
malparty Jun 11, 2021
635d495
root commit
malparty Jun 11, 2021
4b9601f
root commit
malparty Jun 11, 2021
66ee33a
[#7] Update rspec method name change to call
malparty Jun 22, 2021
9d31857
[#7] Use keyword argument to initislize service
malparty Jun 22, 2021
5462897
[#7] Rename GoogleService namespace in Google
malparty Jun 22, 2021
71f477c
[#7] Rename GoogleService namespace in Google missing folder
malparty Jun 22, 2021
9ca34f0
[#7] Update service tests to call with keyword arguments
malparty Jun 22, 2021
ff18ae1
[#7] Update parse_into method to be bang method and return the object
malparty Jun 22, 2021
b1d2594
root commit
malparty Jun 11, 2021
cf86c54
root commit
malparty Jun 11, 2021
e0e1eb4
root commit
malparty Jun 11, 2021
0bf96eb
[#293] Setup DoorKeeper to embbed OAuth2
malparty Jun 9, 2021
f578260
root commit
malparty Jun 11, 2021
1ca527a
[#6] Remove keywords#create action from routes
malparty Jun 22, 2021
847de8f
root commit
malparty Jun 11, 2021
708b704
root commit
malparty Jun 11, 2021
294cf8e
root commit
malparty Jun 11, 2021
5431dac
root commit
malparty Jun 11, 2021
50aeb0a
root commit
malparty Jun 11, 2021
5eed069
root commit
malparty Jun 11, 2021
ec48425
root commit
malparty Jun 11, 2021
1e82221
root commit
malparty Jun 11, 2021
a817125
root commit
malparty Jun 11, 2021
b751213
root commit
malparty Jun 11, 2021
4e8c6d6
root commit
malparty Jun 11, 2021
a4e09d0
root commit
malparty Jun 11, 2021
9ed9447
[#7] Rename GoogleService namespace in Google
malparty Jun 22, 2021
ca3c0a7
[#7] Rename GoogleService namespace in Google missing folder
malparty Jun 22, 2021
6c43759
[#7] Update service tests to call with keyword arguments
malparty Jun 22, 2021
c34874f
[#7] Update client service to match rebase code
malparty Jun 22, 2021
dff824b
[#7] Fix rubocop indentation
malparty Jun 22, 2021
edcb48d
[#7] Remove useless require nokogiri statment
malparty Jun 23, 2021
55783b5
[#7] Use constants to DRY parser css selectors
malparty Jun 23, 2021
438507e
[#7] Rebase from search-raw after user-login-api merge
malparty Jun 23, 2021
aa3a43f
[#7] Rebase from search-raw remove missed file
malparty Jun 23, 2021
3688817
[#7] Remove unwanted form in keyword#index view
malparty Jun 24, 2021
88e9a61
[#7] Fix parser service with call mthod and return hash of attribute
malparty Jun 28, 2021
bdd4d8a
[#7] Rename vcr to simpler google_search/xx name
malparty Jun 28, 2021
070abd5
[#7] Migrate DB after rebase
malparty Jun 28, 2021
e0a42da
[#7] Migrate DB after rebase - fix missing forzen string literal
malparty Jun 28, 2021
9747146
[#26] Add default keyword model
malparty Jun 24, 2021
fd4bc02
[#26] Add soft-delete for users
malparty Jun 28, 2021
d2b455d
[#27] Add UI for keywords#index
malparty Jun 24, 2021
e2c6886
[#27] Fix usage of pagy method
malparty Jun 24, 2021
acc6559
[#27] Add decorator to group by first letter
malparty Jun 24, 2021
f2e8885
[#27] Add style to keyword groups
malparty Jun 24, 2021
b4adae8
[#27] Fix scss declaration order
malparty Jun 24, 2021
7bcb9ba
[#27] Add empty list message
malparty Jun 25, 2021
7076265
[#27] Add UI tests for keywords list
malparty Jun 25, 2021
92a3182
[#27] Simplify render partial views
malparty Jun 25, 2021
0327920
[#27] Use presenter instead of decorator for keywords view helper
malparty Jun 29, 2021
b7ee65f
[#27] Rename keyword presenter into a collection presenter
malparty Jun 29, 2021
37c1d2b
[#27] Singularize css
malparty Jun 29, 2021
8cc1307
[#27] Delete duplicate db migration file after rebase
malparty Jun 30, 2021
a970125
[#27] Delete duplicate table in db schema
malparty Jun 30, 2021
76bd95f
[#27] Fix presenter instance variable accessor
malparty Jun 30, 2021
d8e6042
[#7] Add attr_reader for gogole parser service
malparty Jun 30, 2021
41d05a6
[#7] Wrap specs with describe #method_name
malparty Jun 30, 2021
f5fbcf6
Merge pull request #39 from malparty/feature/google-search-parsed
malparty Jul 1, 2021
86d7c69
Merge pull request #47 from malparty/feature/list-keywords-frontend
malparty Jul 1, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion Gemfile
Expand Up @@ -7,14 +7,15 @@ gem 'pg' # Use Postgresql as database
gem 'puma' # Use Puma as the app server
gem 'mini_magick' # A ruby wrapper for ImageMagick or GraphicsMagick command line
gem 'pagy' # A pagination gem that is very light and fast
gem 'paranoia' # Paranoia is a re-implementation of acts_as_paranoid for Rails 3 and Rails 4. Soft-deletion of records
gem 'discard' # Soft deletes for ActiveRecord
gem 'ffaker' # A library for generating fake data such as names, addresses, and phone numbers.
gem 'fabrication' # Fabrication generates objects in Ruby. Fabricators are schematics for your objects, and can be created as needed anywhere in your app or specs.
gem 'sidekiq' # background processing for Ruby
gem 'bootsnap', require: false # Reduces boot times through caching; required in config/boot.rb
gem 'i18n-js', '3.5.1' # A library to provide the I18n translations on the Javascript
gem 'jsonapi-serializer' # A fast JSON:API serializer for Ruby Objects.
gem 'httparty' # A HTTP client for Ruby.
gem 'nokogiri' # Nokogiri makes it easy and painless to work with XML and HTML from Ruby

# Authentications & Authorizations
gem 'devise' # Authentication solution for Rails with Warden
Expand Down
7 changes: 4 additions & 3 deletions Gemfile.lock
Expand Up @@ -157,6 +157,8 @@ GEM
responders
warden (~> 1.2.3)
diff-lcs (1.4.4)
discard (1.2.0)
activerecord (>= 4.2, < 7)
docile (1.4.0)
doorkeeper (5.5.1)
railties (>= 5)
Expand Down Expand Up @@ -247,8 +249,6 @@ GEM
orm_adapter (0.5.0)
pagy (3.13.0)
parallel (1.20.1)
paranoia (2.4.3)
activerecord (>= 4.0, < 6.2)
parser (3.0.1.1)
ast (~> 2.4.1)
pg (1.2.3)
Expand Down Expand Up @@ -493,6 +493,7 @@ DEPENDENCIES
danger-undercover
database_cleaner
devise
discard
doorkeeper
fabrication
ffaker
Expand All @@ -505,8 +506,8 @@ DEPENDENCIES
letter_opener
listen (= 3.1.5)
mini_magick
nokogiri
pagy
paranoia
pg
pry-byebug
pry-rails
Expand Down
1 change: 1 addition & 0 deletions app/assets/stylesheets/application.scss
Expand Up @@ -15,5 +15,6 @@
@import 'layouts/default';

// Components
@import 'components/list_keywords_group';

// Screens
6 changes: 6 additions & 0 deletions app/assets/stylesheets/components/_list_keywords_group.scss
@@ -0,0 +1,6 @@
.list-keyword-group {
&__header {
min-width: 3em;
color: $gray-500;
}
}
16 changes: 15 additions & 1 deletion app/controllers/keywords_controller.rb
@@ -1,5 +1,19 @@
# frozen_string_literal: true

class KeywordsController < ApplicationController
def index; end
include Pagy::Backend

def index
pagy, keywords_list = pagy(keywords)

render locals: {
pagy: pagy, keywords: KeywordsCollectionPresenter.new(keywords_list)
}
end

private

def keywords
KeywordsQuery.new(current_user).call
end
end
16 changes: 16 additions & 0 deletions app/controllers/registrations_controller.rb
@@ -0,0 +1,16 @@
# frozen_string_literal: true

class RegistrationsController < Devise::RegistrationsController
# Override user hard-delete (from Devise) with user soft-delete
def destroy
malparty marked this conversation as resolved.
Show resolved Hide resolved
malparty marked this conversation as resolved.
Show resolved Hide resolved
resource.discard

Devise.sign_out_all_scopes ? sign_out : sign_out(resource_name)

set_flash_message :notice, :destroyed

yield resource if block_given?

respond_with_navigational(resource) { redirect_to after_sign_out_path_for(resource_name) }
end
end
1 change: 1 addition & 0 deletions app/helpers/application_helper.rb
@@ -1,4 +1,5 @@
# frozen_string_literal: true

module ApplicationHelper
include Pagy::Frontend
end
11 changes: 11 additions & 0 deletions app/models/keyword.rb
@@ -0,0 +1,11 @@
# frozen_string_literal: true

class Keyword < ApplicationRecord
include Discard::Model

validates :name, presence: true, length: { maximum: 255 }

belongs_to :user, inverse_of: :keywords

default_scope -> { kept }
malparty marked this conversation as resolved.
Show resolved Hide resolved
end
6 changes: 6 additions & 0 deletions app/models/user.rb
@@ -1,6 +1,12 @@
# frozen_string_literal: true

class User < ApplicationRecord
include Discard::Model

has_many :keywords, inverse_of: :user, dependent: :destroy

default_scope -> { kept }
malparty marked this conversation as resolved.
Show resolved Hide resolved

# Include default devise modules. Others available are:
# :confirmable, :lockable, :timeoutable, :trackable and :omniauthable
devise :database_authenticatable, :registerable,
Expand Down
15 changes: 15 additions & 0 deletions app/presenters/keywords_collection_presenter.rb
@@ -0,0 +1,15 @@
# frozen_string_literal: true

class KeywordsCollectionPresenter
def initialize(keywords)
@keywords = keywords
end

def groups
keywords.group_by { |keyword| keyword.name[0].upcase.to_sym }
end

private

attr_reader :keywords
end
19 changes: 19 additions & 0 deletions app/queries/keywords_query.rb
@@ -0,0 +1,19 @@
# frozen_string_literal: true

class KeywordsQuery
def initialize(user)
@keywords = user.keywords
end

def call
order_by_name
end

private

attr_reader :keywords

def order_by_name
keywords.order(:name)
end
end
69 changes: 69 additions & 0 deletions app/services/google/parser_service.rb
@@ -0,0 +1,69 @@
# frozen_string_literal: true

module Google
class ParserService
NON_ADS_RESULT_SELECTOR = 'a[data-ved]:not([role]):not([jsaction]):not(.adwords):not(.footer-links)'
AD_CONTAINER_ID = 'tads'
ADWORDS_CLASS = 'adwords'

def initialize(html_response:)
raise ArgumentError, 'response.body cannot be blank' if html_response.body.blank?

@html = html_response

@document = Nokogiri::HTML.parse(html_response)

# Add a class to all AdWords link for easier manipulation
document.css('div[data-text-ad] a[data-ved]').add_class(ADWORDS_CLASS)

# Mark footer links to identify them
document.css('#footcnt a').add_class('footer-links')
end

# Parse html data and return a hash with the results
def call
{
ads_top_count: ads_top_count,
ads_page_count: ads_page_count,
ads_top_url: ads_top_url,
ads_page_url: ads_page_url,
non_ads_result_count: non_ads_result_count,
non_ads_url: non_ads_url,
total_link_count: total_link_count,
html: html
}
end

private

attr_reader :html, :document

def ads_top_count
document.css("##{AD_CONTAINER_ID} .#{ADWORDS_CLASS}").count
end

def ads_page_count
document.css(".#{ADWORDS_CLASS}").count
end

def ads_top_url
document.css("##{AD_CONTAINER_ID} .#{ADWORDS_CLASS}").map { |a_tag| a_tag['href'] }
end

def ads_page_url
document.css(".#{ADWORDS_CLASS}").map { |a_tag| a_tag['href'] }
end

def non_ads_result_count
document.css(NON_ADS_RESULT_SELECTOR).count
end

def non_ads_url
document.css(NON_ADS_RESULT_SELECTOR).map { |a_tag| a_tag['href'] }
end

def total_link_count
document.css('a').count
end
end
end
32 changes: 20 additions & 12 deletions app/views/devise/passwords/new.html.erb
@@ -1,17 +1,25 @@
<h2>Forgot your password?</h2>
<div class="container">
<div class="row justify-content-center">
<div class="col col-md-8 col-lg-6">
<div class="card">
<section class="card-body">
<h2><%= t('auth.forgot_password') %></h2>

<%= form_for(resource, as: resource_name, url: password_path(resource_name), html: { method: :post }) do |f| %>
<%= form_for(resource, as: resource_name, url: password_path(resource_name), html: { method: :post }) do |f| %>

<%= render "devise/shared/error_messages", resource: resource %>
<%= render 'devise/shared/error_messages', resource: resource %>

<div class="field">
<%= f.label :email %><br />
<%= f.email_field :email, autofocus: true, autocomplete: "email" %>
</div>
<div class="form-group">
<%= f.label :email %><br/>
<%= f.email_field :email, autofocus: true, autocomplete: 'email', class: 'form-control' %>
</div>

<div class="actions">
<%= f.submit "Send me reset password instructions" %>
</div>
<% end %>
<%= f.submit t('auth.btn_reset_password'), class: 'btn btn-primary btn-block' %>
<% end %>

<%= render "devise/shared/links" %>
<%= render 'devise/shared/links' %>
</section>
</div>
</div>
</div>
</div>
12 changes: 12 additions & 0 deletions app/views/keywords/_list_keywords.html.erb
@@ -0,0 +1,12 @@
<section class="list-keyword">
<% if keywords.groups.any? %>
<% keywords.groups.each do |group_key, group_keywords| %>
<%= render 'list_keywords_group', group_key: group_key, group_keywords: group_keywords %>
<% end %>
<div class="d-flex justify-content-around">
<%== pagy_bootstrap_nav(pagy) %>
</div>
<% else %>
<div class="alert alert-light"><%= t('keywords.empty_list') %></div>
<% end %>
</section>
13 changes: 13 additions & 0 deletions app/views/keywords/_list_keywords_group.html.erb
@@ -0,0 +1,13 @@
<div class="d-flex flex-row list-keyword-group mb-2">
<div class="list-keyword-group__header">
<h2 class="mb-0">
<%= group_key %>
</h2>
</div>
<ul class="list-inline mb-0">
<% group_keywords.each do |keyword| %>
<li class="list-inline-item list-keyword-item"><%= keyword.name %></li>
<% end %>
</ul>
</div>
<hr/>
5 changes: 3 additions & 2 deletions app/views/keywords/index.html.erb
@@ -1,2 +1,3 @@
<h1>Keywords#index</h1>
<p>Find me in app/views/keyword/index.html.erb</p>
<div class="container">
<%= render 'list_keywords', keywords: keywords, pagy: pagy %>
</div>
2 changes: 1 addition & 1 deletion config/initializers/devise.rb
Expand Up @@ -90,7 +90,7 @@
# It will change confirmation, password recovery and other workflows
# to behave the same regardless if the e-mail provided was right or wrong.
# Does not affect registerable.
# config.paranoid = true
config.paranoid = true

# By default Devise will store the user in session. You can skip storage for
# particular strategies by setting this option.
Expand Down