Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incompatible character encodings error while rendering results #468

Closed
tiagotex opened this issue Mar 18, 2024 · 1 comment
Closed

Incompatible character encodings error while rendering results #468

tiagotex opened this issue Mar 18, 2024 · 1 comment

Comments

@tiagotex
Copy link

We have noticed some encoding bugs when rendering Blazer queries using a MySQL 8 database.

The error is ActionView::Template::Error incompatible character encodings: UTF-8 and ASCII-8BIT (ActionView::Template::Error)

error

The error seemed similar to #459, which was addressed on fd04dd0, since we are using the trilogy adaptor. However, after switching back to mysql2, the error is still present.

It seems like the issue is with rows returning ASCII-8BIT encoded data, and this is causing the crash in ActionView.

I tried to create a reproduction script but could not reproduce the problem except when querying the current data source.

The previous fixes for similar issues were to force UTF8 encoding, I tried to do a quick monkey patch by changing the base helper https://github.com/ankane/blazer/blob/master/app/helpers/blazer/base_helper.rb#L14

module Blazer
  module BaseHelper
    def blazer_format_value(key, value)
      if value.is_a?(Numeric) && !key.to_s.end_with?("id") && !key.to_s.start_with?("id")
        number_with_delimiter(value)
      elsif value.is_a?(String) && value =~ BLAZER_URL_REGEX
        # see if image or link
        if Blazer.images && (key.include?("image") || BLAZER_IMAGE_EXT.include?(value.split(".").last.split("?").first.try(:downcase)))
          link_to value, target: "_blank", rel: "noopener" do
            image_tag value, referrerpolicy: "no-referrer"
          end
        else
          link_to value, value, target: "_blank", rel: "noopener"
        end
      elsif value.is_a?(String) # This is the condition that we added to override encoding
        value.force_encoding("UTF-8")
      else
        value
      end
    end
  end
end

This fixed the issue, but not sure if this is the desired fix.

Please let me know if you would like me to submit a PR with the fix above.

Blazer: 3.0.3
Rails: 7.1.3.2
Ruby 3.3.0
Database: MySQL 8 with Trilogy adapter

Stacktrace 
Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
  from actionview (7.1.3.2) lib/action_view/buffers.rb:45:in `<<'
  from blazer (3.0.3) app/views/blazer/queries/run.html.erb:164:in `block (2 levels) in __var_www_tap_shared_bundle___ruby_______gems_blazer_______app_views_blazer_queries_run_html_erb___380738227109043872_767120'
  from blazer (3.0.3) app/views/blazer/queries/run.html.erb:151:in `each'
  from blazer (3.0.3) app/views/blazer/queries/run.html.erb:151:in `each_with_index'
  from blazer (3.0.3) app/views/blazer/queries/run.html.erb:151:in `block in __var_www_tap_shared_bundle___ruby_______gems_blazer_______app_views_blazer_queries_run_html_erb___380738227109043872_767120'
  from blazer (3.0.3) app/views/blazer/queries/run.html.erb:149:in `each'
  from blazer (3.0.3) app/views/blazer/queries/run.html.erb:149:in `__var_www_tap_shared_bundle___ruby_______gems_blazer_______app_views_blazer_queries_run_html_erb___380738227109043872_767120'
  from actionview (7.1.3.2) lib/action_view/base.rb:264:in `public_send'
  from actionview (7.1.3.2) lib/action_view/base.rb:264:in `_run'
  from actionview (7.1.3.2) lib/action_view/template.rb:261:in `block in render'
  from actionview (7.1.3.2) lib/action_view/template.rb:549:in `instrument_render_template'
  from actionview (7.1.3.2) lib/action_view/template.rb:249:in `render'
  from actionview (7.1.3.2) lib/action_view/renderer/template_renderer.rb:66:in `block (2 levels) in render_template'
  from actionview (7.1.3.2) lib/action_view/renderer/template_renderer.rb:60:in `block in render_template'
  from actionview (7.1.3.2) lib/action_view/renderer/template_renderer.rb:80:in `render_with_layout'
  from actionview (7.1.3.2) lib/action_view/renderer/template_renderer.rb:59:in `render_template'
  from actionview (7.1.3.2) lib/action_view/renderer/template_renderer.rb:11:in `render'
  from actionview (7.1.3.2) lib/action_view/renderer/renderer.rb:63:in `render_template_to_object'
  from actionview (7.1.3.2) lib/action_view/renderer/renderer.rb:31:in `render_to_object'
  from actionview (7.1.3.2) lib/action_view/rendering.rb:135:in `block in _render_template'
  from actionview (7.1.3.2) lib/action_view/base.rb:291:in `in_rendering_context'
  from actionview (7.1.3.2) lib/action_view/rendering.rb:134:in `_render_template'
  from actionpack (7.1.3.2) lib/action_controller/metal/streaming.rb:256:in `_render_template'
  from actionview (7.1.3.2) lib/action_view/rendering.rb:121:in `render_to_body'
  from actionpack (7.1.3.2) lib/action_controller/metal/rendering.rb:158:in `render_to_body'
  from actionpack (7.1.3.2) lib/action_controller/metal/renderers.rb:141:in `render_to_body'
  from actionpack (7.1.3.2) lib/abstract_controller/rendering.rb:27:in `render'
  from actionpack (7.1.3.2) lib/action_controller/metal/rendering.rb:139:in `render'
  from actionpack (7.1.3.2) lib/action_controller/metal/instrumentation.rb:29:in `block (2 levels) in render'
  from benchmark.rb:313:in `realtime'
  from activesupport (7.1.3.2) lib/active_support/core_ext/benchmark.rb:14:in `ms'
  from actionpack (7.1.3.2) lib/action_controller/metal/instrumentation.rb:29:in `block in render'
  from actionpack (7.1.3.2) lib/action_controller/metal/instrumentation.rb:98:in `cleanup_view_runtime'
  from activerecord (7.1.3.2) lib/active_record/railties/controller_runtime.rb:39:in `cleanup_view_runtime'
  from actionpack (7.1.3.2) lib/action_controller/metal/instrumentation.rb:28:in `render'
  from blazer (3.0.3) app/controllers/blazer/queries_controller.rb:268:in `block (2 levels) in render_run'
  from actionpack (7.1.3.2) lib/action_controller/metal/mime_responds.rb:214:in `respond_to'
  from blazer (3.0.3) app/controllers/blazer/queries_controller.rb:266:in `render_run'
  from blazer (3.0.3) app/controllers/blazer/queries_controller.rb:169:in `run'
  from actionpack (7.1.3.2) lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
  from actionpack (7.1.3.2) lib/abstract_controller/base.rb:224:in `process_action'
  from actionpack (7.1.3.2) lib/action_controller/metal/rendering.rb:165:in `process_action'
  from actionpack (7.1.3.2) lib/abstract_controller/callbacks.rb:259:in `block in process_action'
  from activesupport (7.1.3.2) lib/active_support/callbacks.rb:121:in `block in run_callbacks'
  from sentry-rails (5.16.1) lib/sentry/rails/controller_transaction.rb:17:in `block in sentry_around_action'
  from sentry-ruby (5.16.1) lib/sentry/hub.rb:109:in `block in with_child_span'
  from sentry-ruby (5.16.1) lib/sentry/span.rb:206:in `with_child_span'
  from sentry-ruby (5.16.1) lib/sentry/hub.rb:107:in `with_child_span'
  from sentry-ruby (5.16.1) lib/sentry-ruby.rb:478:in `with_child_span'
  from sentry-rails (5.16.1) lib/sentry/rails/controller_transaction.rb:14:in `sentry_around_action'
  from activesupport (7.1.3.2) lib/active_support/callbacks.rb:130:in `block in run_callbacks'
  from activesupport (7.1.3.2) lib/active_support/callbacks.rb:141:in `run_callbacks'
  from actionpack (7.1.3.2) lib/abstract_controller/callbacks.rb:258:in `process_action'
  from actionpack (7.1.3.2) lib/action_controller/metal/rescue.rb:25:in `process_action'
  from actionpack (7.1.3.2) lib/action_controller/metal/instrumentation.rb:74:in `block in process_action'
  from actionpack (7.1.3.2) lib/action_controller/metal/instrumentation.rb:73:in `process_action'
  from actionpack (7.1.3.2) lib/action_controller/metal/params_wrapper.rb:261:in `process_action'
  from activerecord (7.1.3.2) lib/active_record/railties/controller_runtime.rb:32:in `process_action'
  from actionpack (7.1.3.2) lib/abstract_controller/base.rb:160:in `process'
  from actionview (7.1.3.2) lib/action_view/rendering.rb:40:in `process'
  from actionpack (7.1.3.2) lib/action_controller/metal.rb:227:in `dispatch'
  from actionpack (7.1.3.2) lib/action_controller/metal.rb:309:in `dispatch'
  from actionpack (7.1.3.2) lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
  from actionpack (7.1.3.2) lib/action_dispatch/routing/route_set.rb:32:in `serve'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:51:in `block in serve'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:131:in `block in find_routes'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:124:in `each'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:124:in `find_routes'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:32:in `serve'
  from actionpack (7.1.3.2) lib/action_dispatch/routing/route_set.rb:882:in `call'
  from skylight (6.0.4) lib/skylight/probes/action_dispatch/routing/route_set.rb:10:in `block in call'
  from skylight (6.0.4) lib/skylight/probes/action_dispatch/routing/route_set.rb:10:in `call'
  from railties (7.1.3.2) lib/rails/engine.rb:536:in `call'
  from railties (7.1.3.2) lib/rails/railtie.rb:226:in `public_send'
  from railties (7.1.3.2) lib/rails/railtie.rb:226:in `method_missing'
  from actionpack (7.1.3.2) lib/action_dispatch/routing/mapper.rb:22:in `block in <class:Constraints>'
  from actionpack (7.1.3.2) lib/action_dispatch/routing/mapper.rb:51:in `serve'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:51:in `block in serve'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:131:in `block in find_routes'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:124:in `each'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:124:in `find_routes'
  from actionpack (7.1.3.2) lib/action_dispatch/journey/router.rb:32:in `serve'
  from actionpack (7.1.3.2) lib/action_dispatch/routing/route_set.rb:882:in `call'
  from skylight (6.0.4) lib/skylight/probes/action_dispatch/routing/route_set.rb:10:in `block in call'
  from skylight (6.0.4) lib/skylight/probes/action_dispatch/routing/route_set.rb:10:in `call'
  from singed (0.2.1) lib/singed/rack_middleware.rb:17:in `call'
  from warden (1.2.9) lib/warden/manager.rb:36:in `block in call'
  from warden (1.2.9) lib/warden/manager.rb:34:in `catch'
  from warden (1.2.9) lib/warden/manager.rb:34:in `call'
  from rack (3.0.9.1) lib/rack/tempfile_reaper.rb:20:in `call'
  from rack (3.0.9.1) lib/rack/etag.rb:29:in `call'
  from rack (3.0.9.1) lib/rack/conditional_get.rb:43:in `call'
  from rack (3.0.9.1) lib/rack/head.rb:15:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/http/permissions_policy.rb:36:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/http/content_security_policy.rb:33:in `call'
  from rack-session (2.0.0) lib/rack/session/abstract/id.rb:272:in `context'
  from rack-session (2.0.0) lib/rack/session/abstract/id.rb:266:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/cookies.rb:689:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  from activesupport (7.1.3.2) lib/active_support/callbacks.rb:101:in `run_callbacks'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/callbacks.rb:28:in `call'
  from sentry-rails (5.16.1) lib/sentry/rails/rescued_exception_interceptor.rb:12:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
  from sentry-ruby (5.16.1) lib/sentry/rack/capture_exceptions.rb:28:in `block (2 levels) in call'
  from sentry-ruby (5.16.1) lib/sentry/hub.rb:251:in `with_session_tracking'
  from sentry-ruby (5.16.1) lib/sentry-ruby.rb:391:in `with_session_tracking'
  from sentry-ruby (5.16.1) lib/sentry/rack/capture_exceptions.rb:19:in `block in call'
  from sentry-ruby (5.16.1) lib/sentry/hub.rb:59:in `with_scope'
  from sentry-ruby (5.16.1) lib/sentry-ruby.rb:371:in `with_scope'
  from sentry-ruby (5.16.1) lib/sentry/rack/capture_exceptions.rb:18:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
  from lograge (0.14.0) lib/lograge/rails_ext/rack/logger.rb:18:in `call_app'
  from railties (7.1.3.2) lib/rails/rack/logger.rb:24:in `block in call'
  from logstash-logger (0.26.1) lib/logstash-logger/tagged_logging.rb:5:in `block in tagged'
  from logstash-logger (0.26.1) lib/logstash-logger/tagged_logging.rb:16:in `tagged'
  from logstash-logger (0.26.1) lib/logstash-logger/tagged_logging.rb:5:in `tagged'
  from activesupport (7.1.3.2) lib/active_support/broadcast_logger.rb:240:in `method_missing'
  from railties (7.1.3.2) lib/rails/rack/logger.rb:24:in `call'
  from loggery-gem (87fb35a9b4a3) lib/loggery/metadata/middleware/rack.rb:16:in `block in call'
  from loggery-gem (87fb35a9b4a3) lib/loggery/metadata/store.rb:15:in `with_metadata'
  from loggery-gem (87fb35a9b4a3) lib/loggery/metadata/middleware/rack.rb:14:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/remote_ip.rb:92:in `call'
  from request_store (1.6.0) lib/request_store/middleware.rb:19:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/request_id.rb:28:in `call'
  from skylight (6.0.4) lib/skylight/probes/action_dispatch/request_id.rb:8:in `call'
  from rack (3.0.9.1) lib/rack/method_override.rb:28:in `call'
  from rack (3.0.9.1) lib/rack/runtime.rb:24:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/executor.rb:14:in `call'
  from rack (3.0.9.1) lib/rack/sendfile.rb:114:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/ssl.rb:79:in `call'
  from actionpack (7.1.3.2) lib/action_dispatch/middleware/host_authorization.rb:141:in `call'
  from rack-mini-profiler (3.3.1) lib/mini_profiler.rb:191:in `call'
  from skylight (6.0.4) lib/skylight/middleware.rb:99:in `call'
  from railties (7.1.3.2) lib/rails/engine.rb:536:in `call'
  from puma (6.4.2) lib/puma/configuration.rb:272:in `call'
  from puma (6.4.2) lib/puma/request.rb:100:in `block in handle_request'
  from puma (6.4.2) lib/puma/thread_pool.rb:378:in `with_force_shutdown'
  from puma (6.4.2) lib/puma/request.rb:99:in `handle_request'
  from puma (6.4.2) lib/puma/server.rb:464:in `process_client'
  from puma (6.4.2) lib/puma/server.rb:245:in `block in run'
  from puma (6.4.2) lib/puma/thread_pool.rb:155:in `block in spawn_thread'
@ankane
Copy link
Owner

ankane commented Mar 19, 2024

Hi @tiagotex, thanks for reporting. Let's discuss in #466.

@ankane ankane closed this as completed Mar 19, 2024
@D-system D-system mentioned this issue Mar 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ankane @tiagotex