FDSN download denied by the datacenter

Technical Support
1

On the last days I I’m trying to fetch some data and I get always the following info independently of days I want download.

[2024-06-13 00:27:04,727] - obspy.clients.fdsn.mass_downloader - INFO: Initializing FDSN client(s) for https://data.raspberryshake.org.
[2024-06-13 00:27:04,741] - obspy.clients.fdsn.mass_downloader - INFO: Successfully initialized 1 client(s): https://data.raspberryshake.org.
[2024-06-13 00:27:04,742] - obspy.clients.fdsn.mass_downloader - INFO: Total acquired or preexisting stations: 0
[2024-06-13 00:27:04,743] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Requesting unreliable availability.
[2024-06-13 00:27:05,117] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Successfully requested availability (0.37 seconds)
[2024-06-13 00:27:05,118] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Found 1 stations (1 channels).
[2024-06-13 00:27:05,118] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Will attempt to download data from 1 stations.
[2024-06-13 00:27:05,118] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Status for 1 time intervals/channels before downloading: NEEDS_DOWNLOADING
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - ERROR: Client 'https://data.raspberryshake.org' - Request would result in too much data. Denied by the datacenter. Split the request in smaller parts
HTTP Status code: 413
Detailed response of server:

Request Entity Too Large
Error 413: Request Entity Too Large
maximum number of 4.33M samples exceeded
Usage details are available from /fdsnws/dataselect/1/
Request:
/fdsnws/dataselect/1/query
Request Submitted:
2024-06-12T23:27:05.575429
Service Version:
1.1.2
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Launching basic QC checks...
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Downloaded 0.0 MB [0.00 KB/sec] of data, 0.0 MB of which were discarded afterwards.
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Status for 1 time intervals/channels after downloading: DOWNLOAD_FAILED
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - No station information to download.
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - No data could be downloaded.
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: ============================== Final report
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: 0 MiniSEED files [0.0 MB] already existed.
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: 0 StationXML files [0.0 MB] already existed.
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Acquired 0 MiniSEED files [0.0 MB].
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: Client 'https://data.raspberryshake.org' - Acquired 0 StationXML files [0.0 MB].
[2024-06-13 00:27:05,398] - obspy.clients.fdsn.mass_downloader - INFO: Downloaded 0.0 MB in total.

I’m using OBSPY 1.4.1 and it’s the same message whether I use Client("RASPISHAKE") or Client("https://data.raspberryshake.org").

1 Like
2

Hello Fontiela,

This is likely because, as the message states, you have requested too much data in a single request. Try to lower the amount of time you want to retrieve, and I’m sure the query will be successfully completed.

I just tried with 3 and 6-hour requests, and everything appeared to be fine, so the system is working as expected.

3

Hi Stormchaser,

On the exemple I sent, I tried to download only 24h of a single station. I need download several days from multiple stations. In this situation, how can I download one week of data from 10 stations?

2 Likes
4

The current implemented usage rules can be found on the FDSN dedicated page: Raspberry Shake FDSNWS - DataSelect

With your situation, which can definitely be considered a large data request, I would create an automated loop that downloads data in 12-hour batches for the channels of the Shakes you are interested in, advancing until the 10 stations and the entire week have been downloaded.

Rest assured that your feedback on the service has been received, and I have passed it on to our team so that they can discuss what could be done from our side without compromising our system infrastructure.

5

Hello Fontiela,

To update you as promised, we are now upgrading our query capabilities to be able to accommodate entire 24h data requests from FDNSWS, without the need to split it into two 12h tranches.

When you have the chance, could you try to see if your original download program works as intended now?

Thank you for your patience.

6

Hi Stormchaser,

Over the past few days, I haven’t had the time to test the 24-hour data download requests. I tested it today, and it’s running smoothly.

Thank you so much for your support.

2 Likes
7

Happy to read this Fontiela!

Glad we’ve been able to help!