Skip to content

Convert Silixa iDAS TDSM data to MiniSeed and other seismological formats

License

Notifications You must be signed in to change notification settings

pyrocko/idas-convert

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DAS Convert

Convert and downsample distribute acoustic sensing (DAS) data acquired by Silixa iDAS to seismological data formats. Main purpose is to quickly convert and downsample massive amounts of high-resolution TDMS data to MiniSeed.

To handle the massive amount of data generated by DAS interrogators, the conversion tool is leveraging parallel IO and signal-processing processing. On production systems a throughput of 200 MB/s can be archived while converting and downsampling (from 1 kHz to 200 Hz).

The signal processing routines are based on the Pyrocko, mature and well tested seismological framework.

Installation

python3 setup.py install

Documentation

Find the online documentation at https://pyrocko.org/idas_convert/docs/.

Usage

Dump a config file

das_convert dump_config

Example Config

--- !idas.iDASConvertConfig
# Loading TDMS in parallel and process
nthreads_loading: 1
nthreads_processing: 8
queue_size: 32
processing_batch_size: 8

# Threads used for downsampling the data
nthreads: 8

# Input paths
paths:
- /home/isken/src/idas-convert

# Out path, see pyrocko.io for details
outpath: '%(tmin_year)s%(tmin_month)s%(tmin_day)s/%(network)s.%(station)s_%(tmin_year)s%(tmin_month)s%(tmin_day)s.mseed'

# Overwrite mseed meta information
new_network_code: ID
new_channel_code: HSF

downsample_to: 200.0

# MiniSeed record length
record_length: 4096
# MiniSeed STEIM compression
steim: 2

plugins:

# A plugin handling the communication with the GFZ tage file system
- !das_convert.gfz_tapes.GFZTapesConfig
  enabled: false
  bytes_stage: 1T
  waterlevel: 0.6
  wait_warning_interval: 600.0
  release_files: true
  path_tapes_mount: /projects/ether/
  path_tapes_prefix: /archive_FO1/RAW/

# A Telegram bot to keep up-to-date with the process
- !das_convert.telegram_bot.TelegramBotConfig
  enabled: false
  # Telegram API Token
  token: 9e98b8c0567149eb861838a1d770be7d
  # Telegram Chat ID
  chat_id: -1237123123
  # A status message will be dispatched every 3600 s
  status_interval: 3600.0

Start Conversion

das_convert my_config.yml

See das_convert -h for more options.

Citation

DAS Convert - Convert distributed acoustic sensing data, Isken, Marius Paul; Christopher, Wollin; Heimann, Sebastian; Javier, Quinteros; Karl-Heinz, Jäckel; Philippe, Jousset (2021): DAS Convert - Convert distributed acoustic sensing data. V. 1.0. GFZ Data Services. https://doi.org/10.5880/GFZ.2.1.2021.005

Legal Stuff

Licence: GNU General Public License, Version 3, 29 June 2007

Copyright © 2021 Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences, Potsdam, Germany

DAS Convert is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. DAS Convert is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.

About

Convert Silixa iDAS TDSM data to MiniSeed and other seismological formats

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published