Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Yandex translator option to translate_missing functionality #343

Merged
merged 5 commits into from Mar 10, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
22 changes: 22 additions & 0 deletions README.md
Expand Up @@ -104,6 +104,17 @@ $ i18n-tasks translate-missing --backend=deepl
$ i18n-tasks translate-missing --backend=deepl --from=en fr nl
```

### Yandex Translate missing keys

Translate missing values with Yandex Translate ([more below on the API key](#yandex-translation-config)).

```console
$ i18n-tasks translate-missing --backend=yandex

# accepts from and locales options:
$ i18n-tasks translate-missing --from=en es fr
```

### Find usages

See where the keys are used with `i18n-tasks find`:
Expand Down Expand Up @@ -397,6 +408,17 @@ translation:
deepl_api_key: <Deep Pro API key>
```

<a name="yandex-translation-config"></a>
### Yandex Translate

`i18n-tasks translate-missing` requires a Yandex API key, get it at [Yandex](https://tech.yandex.com/translate).

```yaml
# config/i18n-tasks.yml
translation:
yandex_api_key: <Yandex API key>
```

## Interactive console

`i18n-tasks irb` starts an IRB session in i18n-tasks context. Type `guide` for more information.
Expand Down
7 changes: 7 additions & 0 deletions config/locales/en.yml
Expand Up @@ -101,6 +101,13 @@ en:
in config/i18n-tasks.yml. Get the key at https://code.google.com/apis/console.
no_results: >-
Google Translate returned no results. Make sure billing information is set at https://code.google.com/apis/console.
yandex_translate:
errors:
no_api_key: >-
Set Yandex API key via YANDEX_API_KEY environment variable or translation.yandex_api_key
in config/i18n-tasks.yml. Get the key at https://tech.yandex.com/translate.
no_results: >-
Yandex returned no results.
health:
no_keys_detected: No keys detected. Check data.read in config/i18n-tasks.yml.
inconsistent_interpolations:
Expand Down
7 changes: 7 additions & 0 deletions config/locales/ru.yml
Expand Up @@ -101,6 +101,13 @@ ru:
no_results: >-
Google Translate не дал результатов. Убедитесь в том, что платежная информация добавлена
в https://code.google.com/apis/console.
yandex_translate:
errors:
no_api_key: >-
Установить ключ API Яндекса с помощью переменной среды YANDEX_API_KEY или translation.yandex_api_key
в config / i18n-tasks.yml. Получите ключ по адресу https://tech.yandex.com/translate/.
no_results: >-
Яндекс не дал результатов.
health:
no_keys_detected: Ключи не обнаружены. Проверьте data.read в config/i18n-tasks.yml.
inconsistent_interpolations:
Expand Down
1 change: 1 addition & 0 deletions lib/i18n/tasks/configuration.rb
Expand Up @@ -59,6 +59,7 @@ def translation_config
conf = (config[:translation] || {}).with_indifferent_access
conf[:google_translate_api_key] = ENV['GOOGLE_TRANSLATE_API_KEY'] if ENV.key?('GOOGLE_TRANSLATE_API_KEY')
conf[:deepl_api_key] = ENV['DEEPL_AUTH_KEY'] if ENV.key?('DEEPL_AUTH_KEY')
conf[:yandex_api_key] = ENV['YANDEX_API_KEY'] if ENV.key?('YANDEX_API_KEY')
conf
end
end
Expand Down
5 changes: 4 additions & 1 deletion lib/i18n/tasks/translation.rb
Expand Up @@ -2,19 +2,22 @@

require 'i18n/tasks/translators/deepl_translator.rb'
require 'i18n/tasks/translators/google_translator.rb'
require 'i18n/tasks/translators/yandex_translator.rb'

module I18n::Tasks
module Translation
# @param [I18n::Tasks::Tree::Siblings] forest to translate to the locales of its root nodes
# @param [String] from locale
# @param [:deepl, :google] backend
# @param [:deepl, :google, :yandex] backend
# @return [I18n::Tasks::Tree::Siblings] translated forest
def translate_forest(forest, from:, backend: :google)
case backend
when :deepl
Translators::DeeplTranslator.new(self).translate_forest(forest, from)
when :google
Translators::GoogleTranslator.new(self).translate_forest(forest, from)
when :yandex
Translators::YandexTranslator.new(self).translate_forest(forest, from)
else
fail CommandError, "invalid backend: #{backend}"
end
Expand Down
61 changes: 61 additions & 0 deletions lib/i18n/tasks/translators/yandex_translator.rb
@@ -0,0 +1,61 @@
# frozen_string_literal: true

require 'i18n/tasks/translators/base_translator'

module I18n::Tasks::Translators
class YandexTranslator < BaseTranslator
def initialize(*)
begin
require 'yandex-translator'
rescue LoadError
raise ::I18n::Tasks::CommandError, "Add gem 'yandex-translator' to your Gemfile to use this command"
end
super
end

protected

def translate_values(list, **options)
list.map { |item| translator.translate(item, options) }
Copy link
Owner

@glebm glebm Mar 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This is going to cause lots of requests to the Yandex API, processed in-sequence (slow).

It would be better to pass multiple text parameters to the request, batched to avoid hitting the 10,000 character limit, each batch in its own Thread or Fiber. This should be implemented in the yandex gem.

This is not a blocker for submitting this PR (just waiting for a new release of the yandex gem).

end

def options_for_translate_values(from:, to:, **options)
options.merge(
from: to_yandex_compatible_locale(from),
to: to_yandex_compatible_locale(to)
)
end

def options_for_html
{}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Yandex API needs format: 'html' for translating HTML values
https://tech.yandex.com/translate/doc/dg/reference/translate-docpage/

I see that the yandex-translate gem doesn't support the format option yet.
We should make sure HTML keys are supported before merging this, because otherwise they will have broken markup.

Looks like there is an abandoned PR for this aegorov/yandex-translator#13

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened a pr to do that work over there and updated the implementation here to utilize it:
aegorov/yandex-translator#14

end

def options_for_plain
{}
end

def no_results_error_message
I18n.t('i18n_tasks.yandex_translate.errors.no_results')
end

private

# Convert 'es-ES' to 'es'
def to_yandex_compatible_locale(locale)
return locale unless locale.include?('-')
locale.split('-', 2).first
end

def translator
@translator ||= Yandex::Translator.new(api_key)
end

def api_key
@api_key ||= begin
key = @i18n_tasks.translation_config[:yandex_api_key]
fail ::I18n::Tasks::CommandError, I18n.t('i18n_tasks.yandex_translate.errors.no_api_key') if key.blank?
key
end
end
end
end
2 changes: 1 addition & 1 deletion lib/i18n/tasks/version.rb
Expand Up @@ -2,6 +2,6 @@

module I18n
module Tasks
VERSION = '0.9.30'
VERSION = '0.10.1'
glebm marked this conversation as resolved.
Show resolved Hide resolved
end
end