Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible to get available dates from a TimeDimension? #173

Closed
romran-altic opened this issue Apr 10, 2024 · 3 comments
Closed

Is it possible to get available dates from a TimeDimension? #173

romran-altic opened this issue Apr 10, 2024 · 3 comments
Assignees
Labels

Comments

@romran-altic
Copy link

Hello, I am trying to get dimensions explanations with:
cl = structure.dimensions.get(dimension).local_representation.enumerated
sdmx.to_pandas(cl)
Is there any possible approach to get available dates from TimeDimension because code above returns empty.
Asking because I need to get min max date or available dates list from smdx metadata.
Thanks for your help :)

@khaeru khaeru added question needs-info Needs more info from the issuer to proceed labels Apr 10, 2024
@khaeru
Copy link
Owner

khaeru commented Apr 10, 2024

Hi there!

This is a general SDMX question, and doesn't have much to do with this package. It's also hard to answer because you don't say which particular data provider and structures you are working with. For instance, where do you get structure and dimension from? This is necessary info because each data provider chooses different ways to use the SDMX standard to represent their data.

In general—again, these are all features of the SDMX standard or ways it is commonly used, nothing about this package:

  • The Representation (of a dimension in a data structure) can be either enumerated or non-enumerated.
    • "Enumerated" means "only values appearing in a specific code list"—it seems like you're aware fo this.
    • "Non-enumerated" data has no associated code list. It can still have specific formats. These are specified using Facet and related classes. For example, a non-enumerated representation can be specified as "a string with format YYYY-MM", or something like that.
  • Many data providers use enumerated representations for most dimensions, yet not for TIME_PERIOD or similar. This would imply there is no codelist that gives allowable values of that dimension.
  • I think it is possible to use SDMX's constraints mechanisms to specify allowable (a list, or min/max) values of a non-enumerated dimension. But I say "I think" because this is not very common; I don't think most of the known data providers do this.

But again:

@romran-altic
Copy link
Author

Hello, many thanks for your detailed answer. Currently I am trying to get PRC_HICP_CMON table from ESTAT source.
table_msg = self.sdmx_client.dataflow(table)
table_flow = table_msg.dataflow[table]
structure = table_flow.structure

As mentioned for TIME_PERIOD component with structure.dimensions.get('TIME_PERIOD').local_representation I could get Facet:

<Representation: None, [Facet(type=FacetType(is_sequence=None, min_length=None, max_length=None, min_value=None, max_value=None, start_value=None, end_value=None, interval=None, time_interval=None, decimals=None, pattern=None, start_time=None, end_time=None, sentinel_values=None), value=None, value_type=<FacetValueType.observationalTimePeriod: 18>)]>

I noticed value_type=<FacetValueType.observationalTimePeriod: 18>)]> my understanding that it contains 18 values. Could you know if it's possible to retrieve any data from this class?

@khaeru
Copy link
Owner

khaeru commented Apr 10, 2024

Thanks for the extra information. So indeed this is a typical DSD in which TIME_PERIOD is not enumerated by any code list.

I noticed value_type=<FacetValueType.observationalTimePeriod: 18>)]> my understanding that it contains 18 values.

No, that's not correct. FacetValueType is an ordinary Python Enum; you can see this in the documentation where it says "Bases: Enum". For instance:

>>> from enum import Enum
>>> Foo = Enum("Foo", "A B C")
>>> Foo.C
<Foo.C: 3>

The only thing we can infer from "18" is that "observationalTimePeriod" is the 18th entry in this list:

sdmx/sdmx/model/common.py

Lines 478 to 496 in 8d5e467

#: SDMX FacetValueType.
#:
#: In the SDMX 2.0 IM, three diagrams in the spec show this enumeration containing
#: 'gregorianYearMonth' but not 'gregorianYear' or 'gregorianMonth'. The table in
#: §3.6.3.3 Representation Constructs does the opposite. One ESTAT query (via SGR) shows
#: a real-world usage of 'gregorianYear'; while one query shows usage of
#: 'gregorianYearMonth'; so all three are included.
FacetValueType = Enum(
"FacetValueType",
"""string bigInteger integer long short decimal float double boolean uri count
inclusiveValueRange alpha alphaNumeric numeric exclusiveValueRange incremental
observationalTimePeriod standardTimePeriod basicTimePeriod gregorianTimePeriod
gregorianYear gregorianMonth gregorianYearMonth gregorianDay reportingTimePeriod
reportingYear reportingSemester reportingTrimester reportingQuarter reportingMonth
reportingWeek reportingDay dateTime timesRange month monthDay day time duration
keyValues identifiableReference dataSetReference """
# SDMX 3.0 only
"geospatialInformation",
)

Again, this FacetValueType is directly copied from the SDMX standards. Please, click on "Section 2 — Information Model" and search for "FacetValueType"; you will find it described in Section 3.6 of that file.

@khaeru khaeru removed the needs-info Needs more info from the issuer to proceed label Apr 10, 2024
@khaeru khaeru changed the title Get available dates from TImeDimension Get available dates from TimeDimension May 22, 2024
@khaeru khaeru changed the title Get available dates from TimeDimension Is it possible to get available dates from a TimeDimension? May 22, 2024
@khaeru khaeru self-assigned this May 22, 2024
@khaeru khaeru closed this as completed May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants