Data not available for several sensors when try to download

Technical Support
1

Hi there,

I tried to download the data from New Zealand. Even though my sensors are online, I couldn’t download the data for an entire day. Size of each packets ENZ/ ENN/ ENE seems to be different. What is the reason behind this? It would be great if someone can help on this since it is for some research purpose.

Thanks in advance.

2

Hello chanthuj,

We are experiencing issues that prevent data from being displayed or downloaded on/from some of our services such as fdsnws and dataview.

We are working to identify and fix the problem as soon as possible. I will update you when everything has been solved.

Thank you for your patience and apologies for any inconvenience.

4

Thanks so much. It would be a great help to our team. We are planning to download 48 hours of data continuous for some research purposes.

5

Helklo chanthuj,

I have been informed that the services have been restored in full, and everything should be working fine now. You can try again to download the data you need for your research.

If the packet sizes appear different, it is possible that some of the units, or some of the sensors, did not transmit data to our servers for a number of reasons, so you will have to choose a different timeframe to have the uninterrupted 48h dataset.

You can check on our DataView service (https://dataview.raspberryshake.org/) before downloading, so that you will have an idea on what data is missing (if there is any).

1 Like
6

Thanks for your effort. However, I couldn’t download data from any of the sensors for more than 15 mins on any given day. I tried downloading the data using 2 min data block, which was working fine, but the results were not continuous. Data is missing here and there in all the sensors. Could you please let me know a way to download continuous data? All the sensors are connected to the internet perfectly all the time.

Tested sensors: RD9AE, R9229, RACC4

7

Hello chanthuj,

Could I ask you for the code you are using to download the data?

I have tried some random dates/times via our FDSNWS system, and data is downloaded correctly. For example, these times provide 15 minutes of data from RD9AE: https://fdsnws.raspberryshakedata.com/fdsnws/dataselect/1/query?starttime=2022-11-02T20%3A30%3A00&endtime=2022-11-02T21%3A00%3A00&network=AM&station=RD9AE&location=00&channel=EHZ&nodata=404

You can also download the data directly from the RaspberryShake (if you have access to them) via SSH or other means. We have a dedicated page here on our manual: How to download your data — Instructions on Setting Up Your Raspberry Shake

If you wish, you can also send me the logs from your Shake(s) so that I can try to see if there is any issue that is potentially causing what you are seeing. Instructions on how to download them, if needed, are here: Please read before posting!

8

Thanks for your reply. Please find my code below. It would be great if you can help with that.

Please find my code used below,

from future import print_function
from obspy import UTCDateTime
from obspy.clients.fdsn import Client
import matplotlib
import os

matplotlib.use(‘TkAgg’)

station_list = [‘RD9AE’]

network = “NZ”
location = “*”
channel = “?N?” # or “HHZ”, “??E”, “*N” etc.
output = “ACC” # also"VEL" “DISP” or “ACC”

def create_dir(dir):
if not os.path.exists(dir):
os.makedirs(dir)
return dir

start_1 = “2022-10-11 02:00:00”
end = “2022-10-11 03:00:00”
starttime = UTCDateTime(start_1)
endtime = UTCDateTime(end)

c = Client(“RASPISHAKE”)

for stn in station_list:
try:
st = c.get_waveforms(‘AM’, stn, ‘00’, ‘EN?’, starttime, endtime, attach_response=True)
st.remove_response(output=output)
st.trim(starttime=starttime, endtime=endtime)
st.sort()
# write station_name in a txt file
if (len(st[0]) == len(st[1]) == len(st[2])):
path = “Path to the Folder”+ “Data_Contiuous” + “/”
create_dir(path)
print(“Path Created”)
print("Data saved for station: " + stn)
print(“Data start time”+str(st[0].stats.starttime))
print(“Data start time” + str(st[0].stats.endtime))
for tr in st:
path_temp = ‘/Users/chanthujan/PycharmProjects/RaspberryShake/RPI_Data/’ + “Data_Contiuous” + “/” + stn
create_dir(path_temp)
filename = path_temp + ‘/’ + tr.stats.channel + “_” + “DATA” + “.mseed”
tr.write(filename, format=“MSEED”)
else:
print(“print the length of the streams” + str(len(st[0])) + " " + str(len(st[1])) + " " + str(len(st[2])))
except Exception as e:
print(“failed, for reason:\n{}”.format(e))
print(“failed station–” + str(stn))
continue

1 Like
9

Hello chanthuj,

Thank you for your code. The first annotation is that the network value is not correct, as all the RaspberryShake stations are in the AM network. It’s an easy adjustment to do in any case.

With this modification, the streams (st) are downloaded without issues for all channels of all three stations with the iteration process you have set up. However, some streams present interruptions in them, which can be due to a range of reasons which I cannot be sure of without taking a look at the logs of all Shakes.

However, you can overcome this obstacle by merging the streams together, which will create a single stream of the length you have specified (in this case, 1 hour, so 3600 seconds, and by 100 samples per second, 360000 samples).

This is the ObsPy command to execute for each stream:

st.merge(method=0, fill_value=None)

and this is the page that explains the method: obspy.core.stream.Stream.merge — ObsPy 1.3.1 documentation

Using this, I get, for station RD9AE, for example:

1 Trace(s) in Stream:
AM.RD9AE.00.ENE | 2022-10-11T02:00:00.003000Z - 2022-10-11T03:00:00.003000Z | 100.0 Hz, 360001 samples
1 Trace(s) in Stream:
AM.RD9AE.00.ENZ | 2022-10-11T02:00:00.003000Z - 2022-10-11T03:00:00.003000Z | 100.0 Hz, 360001 samples
1 Trace(s) in Stream:
AM.RD9AE.00.ENN | 2022-10-11T02:00:00.003000Z - 2022-10-11T03:00:00.003000Z | 100.0 Hz, 360001 samples
10

Hi,

Thanks for your reply. I will check on this. However, you have mentioned that the network value I have used is incorrect. Could you please let me know what value I need to set?

Thanks

1 Like
11

Hello chanthuj,

Sure, there is no problem at all. All our Shakes in the RaspberryShake network can be found with the AM code. So you just need to set

network = "AM"

and you’re ready to go.

12

Thanks so much for you support.

2 Likes
13

You’re very welcome.