Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#6 As a user, I can query a single keyword and get its Google search raw response #36

Merged
merged 47 commits into from Jun 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
b402a7f
[#6] Add google client service
malparty Jun 14, 2021
dbf2e9d
[#6] Add basic create action and form to perform a google query
malparty Jun 14, 2021
89a9645
[#6] Replace obsolete URI.encode by CGI.escape
malparty Jun 14, 2021
27fbc17
[#6] Add tests using VCR to replay a google search query
malparty Jun 14, 2021
ccb4c85
[#5] Generate Devise views
malparty Jun 9, 2021
b180494
[#5] Add lastname and firstname to registration views
malparty Jun 9, 2021
3c0b3d3
Rebase from develop routes
malparty Jun 16, 2021
f430136
Rebase from develop pull
malparty Jun 16, 2021
18c3631
Rebase from develop pull
malparty Jun 16, 2021
a2d7d94
root commit
malparty Jun 11, 2021
36900b1
[#5] Rename lastname/firstname into last_name first_name
malparty Jun 14, 2021
9de1276
[#5] Fix missing renaming for lastname firstname
malparty Jun 14, 2021
5184e73
[#5] Merge email (citext) and renaming names into previous db migrations
malparty Jun 14, 2021
8b3ffc0
[#6] Declare vcr inline to improve code readability
malparty Jun 16, 2021
11f09cd
[#6] Replace instance variables by local variables inside view
malparty Jun 16, 2021
f60ba32
[#6] Handle error while query google - not colorized
malparty Jun 16, 2021
2441302
[#6] Review methods order and exposure
malparty Jun 16, 2021
1af4e64
[#6] Update tests from class method to instance methods
malparty Jun 16, 2021
0236a3a
[#6] Colorize logs from Rails.logger
malparty Jun 16, 2021
5a52417
root commit
malparty Jun 11, 2021
67171f8
root commit
malparty Jun 11, 2021
242c5db
[#6] Improve create action readability
malparty Jun 17, 2021
c74da8a
[#6] Remove create action and index form
malparty Jun 21, 2021
d289a9b
[#6] remove useless require
malparty Jun 21, 2021
603ac8e
[#6] Set base url google as constant
malparty Jun 21, 2021
b2f1d52
[#6] Fix formatting warnings
malparty Jun 21, 2021
d233c30
[#6] Rename query_result in call
malparty Jun 21, 2021
8ace4ab
[#6] Use keyword arguments to initialize service
malparty Jun 22, 2021
f409671
[#6] Rename GoogleService namespace in Google
malparty Jun 22, 2021
a50ea52
[#6] Rename GoogleService namespace in Google
malparty Jun 22, 2021
73dac87
[#6] Fix test to map keyword arguments
malparty Jun 22, 2021
dbaa2dd
[#6] Review implementation of ClientService.call to be less risky wit…
malparty Jun 22, 2021
3b993d8
[#6] Fix rubocop empty line
malparty Jun 22, 2021
4f4805c
[#6] Fix nil keyword in error messages
malparty Jun 22, 2021
f9d9607
[#6] Fix rubocop warnings
malparty Jun 22, 2021
36c0d94
[#6] Improve tests to cover result validation
malparty Jun 22, 2021
500bda9
[#6] Improve tests to cover result validation - spelling fix
malparty Jun 22, 2021
44ef5d2
[#6] Improve call method for worker retrial ease
malparty Jun 22, 2021
ab621cc
[#6] Rename validate_result in valid_result?
malparty Jun 22, 2021
5d8ec49
Update Gemfile Httparty comment
malparty Jun 23, 2021
dd4b0f8
Replace nil by false in clientservice spec
malparty Jun 23, 2021
0e68db9
[#6] Update client service spec with const google url instead of hard…
malparty Jun 23, 2021
902742d
[#6] Update client service valid_result? method to return true when v…
malparty Jun 23, 2021
93704d4
[#6] Replace falsy by eq(false) in client service spec test
malparty Jun 23, 2021
4c5bed3
[#6] Remove optional comment
malparty Jun 23, 2021
777e61b
[#6] Rebase from develop had wrong schema version
malparty Jun 23, 2021
8dbecbc
Merge branch 'develop' into feature/google-search-raw
malparty Jun 23, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions Gemfile
Expand Up @@ -14,6 +14,7 @@ gem 'sidekiq' # background processing for Ruby
gem 'bootsnap', require: false # Reduces boot times through caching; required in config/boot.rb
gem 'i18n-js', '3.5.1' # A library to provide the I18n translations on the Javascript
gem 'jsonapi-serializer' # A fast JSON:API serializer for Ruby Objects.
gem 'httparty' # A HTTP client for Ruby.

# Authentications & Authorizations
gem 'devise' # Authentication solution for Rails with Warden
Expand All @@ -24,6 +25,9 @@ gem 'doorkeeper' # Awesome OAuth 2 provider for your Rails / Grape app
gem 'webpacker', '~>5.2.0' # Transpile app-like JavaScript
gem 'sass-rails' # SASS

# Logging tools
gem 'colorize' # Ruby gem for colorizing text using ANSI escape sequences

# Translations
# gem 'devise-i18n' # Translations for Devise
# gem 'rails-i18n', '~> 6.0.0' # Translations for Rails
Expand Down
10 changes: 10 additions & 0 deletions Gemfile.lock
Expand Up @@ -97,6 +97,7 @@ GEM
sexp_processor
coderay (1.1.3)
colored2 (3.1.2)
colorize (0.8.1)
concurrent-ruby (1.1.9)
connection_pool (2.2.5)
cork (0.3.0)
Expand Down Expand Up @@ -188,6 +189,9 @@ GEM
globalid (0.4.2)
activesupport (>= 4.2.0)
hashdiff (1.0.1)
httparty (0.18.1)
mime-types (~> 3.0)
multi_xml (>= 0.5.2)
i18n (1.8.10)
concurrent-ruby (~> 1.0)
i18n-js (3.5.1)
Expand Down Expand Up @@ -220,11 +224,15 @@ GEM
mini_mime (>= 0.1.1)
marcel (1.0.1)
method_source (1.0.0)
mime-types (3.3.1)
mime-types-data (~> 3.2015)
mime-types-data (3.2021.0225)
mini_magick (4.11.0)
mini_mime (1.0.3)
mini_portile2 (2.5.3)
minitest (5.14.4)
msgpack (1.4.2)
multi_xml (0.6.0)
multipart-post (2.1.1)
nap (1.1.0)
nio4r (2.5.7)
Expand Down Expand Up @@ -473,6 +481,7 @@ DEPENDENCIES
brakeman
bullet
capybara (>= 2.15)
colorize
danger
danger-brakeman_scanner
danger-eslint
Expand All @@ -489,6 +498,7 @@ DEPENDENCIES
ffaker
figaro
foreman
httparty
i18n-js (= 3.5.1)
json_matchers
jsonapi-serializer
Expand Down
40 changes: 40 additions & 0 deletions app/services/google/client_service.rb
@@ -0,0 +1,40 @@
# frozen_string_literal: true

module Google
class ClientService
USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) '\
'AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36'

BASE_SEARCH_URL = 'https://www.google.com/search'

def initialize(keyword:, lang: 'en')
@escaped_keyword = CGI.escape(keyword)
@uri = URI("#{BASE_SEARCH_URL}?q=#{@escaped_keyword}&hl=#{lang}&gl=#{lang}")
end

def call
Copy link

@github-actions github-actions bot Jun 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Has approx 6 statements

result = HTTParty.get(@uri, { headers: { 'User-Agent' => USER_AGENT } })

return false unless valid_result? result

result
rescue HTTParty::Error, Timeout::Error, SocketError => e
Copy link

@github-actions github-actions bot Jun 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Has the variable name 'e'

Rails.logger.error "Error: Query Google with '#{@escaped_keyword}' thrown an error: #{e}".colorize(:red)

false
end

private

# Inspect Http response status code
# Any non 200 response code will be logged
def valid_result?(result)
return true if result&.response&.code == '200'

Rails.logger.warn "Warning: Query Google with '#{@escaped_keyword}' return status code #{result.response.code}"
.colorize(:yellow)

false
end
end
end
2 changes: 2 additions & 0 deletions config/locales/en.yml
Expand Up @@ -33,6 +33,8 @@ en:
app_name: 'Google Search Ruby'
doorkeeper:
token_revoked: 'If your token was valid, it has been revoked'
keywords:
could_not_query: 'An error occurs when performing the Google Search, please try again.'
auth:
logout: 'Sign out'
sign_up: 'Sign up'
Expand Down
85 changes: 43 additions & 42 deletions db/schema.rb

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

112 changes: 112 additions & 0 deletions spec/fixtures/vcr/google_search.yml

Large diffs are not rendered by default.

63 changes: 63 additions & 0 deletions spec/fixtures/vcr/google_warn.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 36 additions & 0 deletions spec/services/google/client_service_spec.rb
@@ -0,0 +1,36 @@
# frozen_string_literal: true

require 'rails_helper'

RSpec.describe Google::ClientService, type: :service do
context 'when querying a simple keyword' do
it 'returns an HTTParty Response', vcr: 'google_search' do
result = described_class.new(keyword: FFaker::Lorem.word).call

expect(result).to be_an_instance_of(HTTParty::Response)
end

it 'queries Google Search', vcr: 'google_search' do
path = described_class.new(keyword: FFaker::Lorem.word).call.request.path

expect(path.to_s).to start_with(described_class::BASE_SEARCH_URL)
end
end

context 'when google returns an HTTP error' do
it 'returns false', vcr: 'google_warn' do
result = described_class.new(keyword: FFaker::Lorem.word).call

expect(result).to eq(false)
end

it 'logs a warning with the escaped keyword', vcr: 'google_warn' do
allow(Rails.logger).to receive(:warn)

word = FFaker::Lorem.word
described_class.new(keyword: word).call

expect(Rails.logger).to have_received(:warn).with(/#{CGI.escape(word)}/)
end
end
end
1 change: 1 addition & 0 deletions spec/support/authentication_helper.rb
Expand Up @@ -16,6 +16,7 @@ def sign_in_ui(user = nil)

fill_in 'user_email', with: user.email
fill_in 'user_password', with: user.password

click_button 'Sign in'
end

Expand Down
2 changes: 1 addition & 1 deletion spec/support/vcr.rb
Expand Up @@ -14,7 +14,7 @@
end

RSpec.configure do |config|
# You can pass a hash to the `vcr` tag to specifcy additional options:
# You can pass a hash to the `vcr` tag to specify additional options:
# vcr: { group: 'places/google/details', cassettes: %w(kfc red_planet)}
# vcr: { group: 'places/google/details', cassette: 'kfc'}
# vcr: { cassette: 'places/google/details', options: { decode_compressed_response: true } }
Expand Down