Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IMF url changed #38

Open
bertrandmarc opened this issue Dec 17, 2020 · 2 comments
Open

IMF url changed #38

bertrandmarc opened this issue Dec 17, 2020 · 2 comments
Labels
data-source Issues related to specific web services/data source(s) needs-info Needs more info from the issuer to proceed

Comments

@bertrandmarc
Copy link

It seems the url for IMF changed. It is now possible to get data from IMF in SDMX 2.0, 2.1 and JSON.
SDMX XML: http://datahelp.imf.org/knowledgebase/articles/1952905-sdmx-2-0-and-sdmx-2-1-restful-web-service
SDMX JSON: http://datahelp.imf.org/knowledgebase/articles/667681-using-json-restful-web-service

Unfortunately, to get SDMX XML 2.1 data, one needs to add a parameter format=sdmx-2.1 to all requests, and I don't know how to add that in the sources list.

@khaeru khaeru added the data-source Issues related to specific web services/data source(s) label Dec 17, 2020
@khaeru
Copy link
Owner

khaeru commented Dec 18, 2020

Great, thanks for the report.

It's interesting: from that page, it looks like the web service API and URL scheme are those of SDMX 2.0. So it's just that query parameter which allows retrieving messages in SDMX-ML 2.1. I haven't seen a hybrid like that before.

The way to handle this is to set the modify_request_args() hook on a source-specific class. For an example see

sdmx/sdmx/source/ilo.py

Lines 7 to 24 in 30d8145

def modify_request_args(self, kwargs):
"""Handle two limitations of ILO's REST service.
1. Service returns SDMX-ML 2.0 by default, whereas :mod:`sdmx` only
supports SDMX-ML 2.1. Set ``?format=generic_2_1`` query parameter.
2. The service does not support values 'parents', 'parentsandsiblings'
(the default), and 'all' for the ``references`` query parameter.
Override the default with ``?references=none``.
.. note:: Valid values are: none, parents, parentsandsiblings,
children, descendants, all, or a specific structure reference
such as 'codelist'.
"""
super().modify_request_args(kwargs)
kwargs.setdefault("params", {})
kwargs["params"].setdefault("format", "generic_2_1")
kwargs["params"].setdefault("references", "none")

@khaeru khaeru added the help welcome Issues that depend on contributions from new developers label Dec 18, 2020
@khaeru khaeru self-assigned this Jan 4, 2021
@khaeru khaeru removed their assignment Jan 19, 2021
@khaeru
Copy link
Owner

khaeru commented Jan 19, 2021

@bertrandmarc on another look, I am not sure if these pages are current.

  • The current base URL is: https://sdmxcentral.imf.org/ws/public/sdmxapi/rest/.
  • Using this produces working queries for all the metadata endpoints; these run every day as part of the sdmx1 test suite.
  • The data endpoint, for example https://sdmxcentral.imf.org/ws/public/sdmxapi/rest/data/CPI/CPI..PCPI_2009_IX., produces a response with an error message (e.g. in this case "Not enough key values in query, expecting 5 got 4"). I have tried a little to construct a working data query to add to the test suite, but haven't been able to.
  • The domain sdmxcentral.imf.org contains the usual IM-JS web explorer, e.g. https://sdmxcentral.imf.org/data/overview.html
  • This is different from the domain dataservices.imf.org in two pages you link.

It's entirely possible that IMF is operating two services. Maybe one is a legacy one that will eventually be retired, and the other is a modern one. It's also possible the documentation has not been kept up to date (there are no dates mentioned on the pages you linked).

If this is an important data source, would you mind to get in touch with them to ask what the situation is? Better than trying to guess.

@khaeru khaeru added needs-info Needs more info from the issuer to proceed and removed help welcome Issues that depend on contributions from new developers labels Jan 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data-source Issues related to specific web services/data source(s) needs-info Needs more info from the issuer to proceed
Projects
None yet
Development

No branches or pull requests

2 participants