Skip to content

Mass Data Processing with SurfQuake

⚠️ Note: This feature is under active development. The expected stable release is SurfQuake 0.0.9 (July 2025).

Welcome to the Mass Processing Module of SurfQuake — a powerful and scalable command-line tool for applying signal processing pipelines to large collections of seismic waveform data.

Whether you're analyzing earthquake catalogs or preparing data for machine learning models, this module allows you to define, control, and automate your signal processing workflow with precision.

📘 Learn more in our Signal Processing Tutorial

Macro Structure Overview

The macro configuration (provided via a YAML file) defines your signal processing pipeline in a structured, step-by-step format. Each process entry specifies a method, its parameters, and the order in which it will be applied to each trace. You can process either full daily waveform files or extract and process specific segments based on an event file. Additionally, an interactive plotting tool is available to help you visually inspect and validate the processing steps.

Running the Process

This tool runs via the terminal:

surfquake processing_cut -h

surfquake processing_cut -p [project_file] -o [output_folder] -i [metadata_file] -c [config_file] \
-e [event_file] -n [network] -s [station] -ch [channel] \
-cs [cut_start] -ce [cut_end] -t [cut_time] -r -l --plot_config [protting_file_config]

surfquake processing_daily -h

surfquake processing_daily -p [project_file] -o [output_folder] -i [metadata_file] --min_date [Start time filter: format 'YYYY-MM-DD HH:MM:SS.sss'] --max_date [End time filter: format 'YYYY-MM-DD HH:MM:SS.sss'] -l --plot_config [protting_file_config] --span_seconds [Time span to search split (in seconds)] -n [net filter] -s [stations filter] -c [channels filter]

Processing Options

Detrending

Remove unwanted trends from the raw signal:

  • linear: Remove a best-fit line
  • demean: Subtract the mean
  • polynomial: Fit and subtract a polynomial (e.g., degree 3)
  • spline: Subtract a smooth spline fit

Tapering

Apply a window to reduce spectral leakage near signal edges.

  • Parameter: max_percentage (e.g., 0.05 for 5%)
  • Supported Windows: cosine, hann, hamming, blackman, kaiser, flattop, slepian, and many more...

Normalize

Rescale amplitude:

  • 0: Normalize by maximum
  • Custom value: Divide signal by provided constant

Filtering

Apply filters to isolate specific frequency content.

  • Parameters: freqmin, freqmax, zerophase, corners (poles)
  • Filter Types:
  • bandpass, lowpass, highpass, bandstop
  • Parametric options: cheby1, cheby2, elliptic, bessel

ℹ️ Ensure freqmin < freqmax for band filters.

Denoising

Wiener Filter
  • Parameters: time_window (seconds), noise_power
  • Tip: If noise_power=0, it’s estimated from local variance.
Wavelet Denoise
  • Parameters: wavelet_family (e.g., sym8), threshold (e.g., 0.05)

Resampling

Adjust the sampling rate (e.g., for standardization):

  • Parameters: new_sampling_rate, pre_filter (recommended: True)

Fill Gaps

  • Methods: interpolate, latest (repeat last value)

Differentiate

Estimate the signal’s derivative:

  • Methods: gradient

Integrate

Reconstruct displacement or velocity:

  • Methods: cumtrapz, spline

Time Shifting

Shift traces forward or backward in time.

  • Parameter: List of time shifts in seconds

Remove Instrument Response

Convert raw counts to physical units (e.g., velocity or displacement):

  • Parameters: pre_filt (list), water_level, units, inventory

Add Noise

White Noise:

  • Parameter: SNR_dB relative to original trace

Colored Noise (optional, not in base config):

  • Use exponent for control: 0 (white), 1 (pink), 2 (brown), -1 (blue), etc.

Spectral Whitening

Flattens signal spectrum while preserving phase:

  • Parameters: freq_width (Hz), taper_edge (True/False)

Time Normalization

Standardize temporal energy distribution:

  • Methods: time-normalization, 1bit, clipping
  • Parameters: norm_win, iterations

Smoothing

Reduce high-frequency noise or jitter:

  • Parameters: time_window, fwhm
  • Methods: mean, gaussian, tkeo

Spike Removal

Removes isolated outliers using hampel algorithm:

  • Parameters: window_size, threshold (in standard deviations)

Example Plotting File Format (YAML)

plotting:
  enabled: true                # override -l CLI flag if needed
  traces_per_fig: 3
  sort_by: distance            # options: 'distance', 'backazimuth', or null
  vspace: 0.1                  # vertical space between subplots
  show_legend: true
  title_fontsize: 9
  autosave: false              # option to save plots instead of just showing
  save_folder: ./plots         # folder to save figures if autosave is true

Example Event File Format

date;hour;latitude;longitude;depth;magnitude
2022-02-02;23:35:29.7;42.5089;1.4293;20.7;1.71
2022-02-03;12:01:21.6;42.3047;2.2741;0.0;1.65

Example Config File (YAML)

Analysis:
  process_1:
    name: 'rmean'
    method: 'linear'
  process_2:
    name: 'taper'
    method: 'cosine'
    max_percentage: 0.05
  process_3:
    name: 'resample'
    sampling_rate: 20
    pre_filter: True
  process_4:
    name: 'fill_gaps'
    method: 'latest'
  process_5:
    name: 'normalize'
    norm: True
  process_6:
    name: 'filter'
    type: 'bandpass'
    fmin: 0.5
    fmax: 8.5
    corners: 4
    zerophase: True
  process_7:
    name: 'add_noise'
    noise_type: 'white'
    SNR_dB: 1
  process_8:
    name: 'wavelet_denoise'
    dwt: 'sym8'
    threshold: 0.1
  process_9:
    name: 'differentiate'
    method: 'spectral'
  process_10:
    name: 'integrate'
    method: 'spectral'
  process_11:
    name: 'whitening'
    freq_width: 0.02
    taper_edge: True
  process_12:
    name: 'smoothing'
    method: 'gaussian'
    time_window: 5
    FWHM: 0.05
  process_13:
    name: "time_normalization"
    method: 'time normalization'
    norm_win: 10
  process_14:
    name: "wiener_filter"
    time_window: 1.0
    noise_power: 0
  process_15:
    name: 'rotate'
    method: 'GAC'