Hello,
I’ve been running an old shake for years now but recently noticed it was no longer connected to the server. I did some basic troubleshooting but my skills have left the building. I do see a connection error but the error code is strange:
2021 341 14:41:27>> Connection request (raspberryshakedata.com:55555) failed with error code: Operation now in progress
myshake@raspberryshake:/opt/log $ ping raspberryshake.com
PING raspberryshake.com (82.192.82.227) 56(84) bytes of data.
64 bytes from 82.192.82.227 (82.192.82.227): icmp_seq=11 ttl=51 time=112 ms
64 bytes from 82.192.82.227 (82.192.82.227): icmp_seq=12 ttl=51 time=111 ms
I’ve tried to figure it out looking at other posts but to no avail. I have no idea when it stopped uploading, local swarm works fine.
Any ideas please?
Thanks RSH.RC345.2022-01-10T00_40_52.logs.tar (3.5 MB)
Yes, it is indeed an unusual error. Could you please shut down the Shake, wait for around 5 minutes, then turn it on again, wait for another 30 min, download the new logs and post them here?
I want to observe if there is a similar (or, literally, the same) behavior after a fresh start of the station.
Thank you for the help!
After powering off and waiting five minutes, waited another 30 mins and here are the logs.
Cheers! RSH.RC345.2022-01-10T19_28_39.logs.tar (3.5 MB)
Thank you for the new logs. It appears that the problem is still there, however, this time the Shake manages to find a time synchronization server (NTP), so that is a step forward.
Could you please reboot the Shake and then try the following procedure to see if it solves the connectivity issue?
Please access your rs.local/ page, go to Settings (the gear icon high on the left), and then the Data tab. Make sure that the Forward Data box is checked, and then click Save and Restart .
The station should now be able to connect again.
If it does, then this error is caused by a known bug that we are examining and that will (hopefully) be solved in the next Shake OS v0.20 release.
I checked data forward was on, I actually unchecked it, then rechecked it and hit save. Waited a few minutes and still not connected. I rebooted and waited another 5 minutes, still no connection but here are the logs.
Thank you again! RSH.RC345.2022-01-11T21_24_13.logs.tar (3.6 MB)
Thank you for the new logs and the further tests Wang.
It seems, that, compared to before, when the Shake managed to successfully connect to our servers, something has changed in the local network, with the results that we are seeing.
Could you please check that if the following ports
port 55555 [TCP]
port 55556 [TCP]
are open, and if not, please open them on your modem/router/network.
In any case, also check that
port 123 for [TCP] and [UDP] traffic in both directions
is also open. This last one is the port needed for the NTP time service, and it is required to keep the data synchronized with our network.
R5A78 here, same or similar issue. I would note that I also upload raw data to a computer in San Jose and I see that data as expected giving me some confidence that the network connection is functional,RSH.R5A78.2022-01-13T18_31_20.logs.tar (4.5 MB)
Has this not yielded any positive results, and the Shake still doesn’t connect to our servers?
Another element that I noticed is that in the Nameserver list, the only address that is present is a local one, 127.0.0.1, which is the address for loopback localhost traffic.
Could you please shut down the Shake, wait for around 5 minutes, then turn it on again, wait for another 30 min, download the new logs and post them here?
As far as I can tell, the relevant ports are open and I do see data correctly via
datacast (bits are making it as expected from Hawaii to San Jose) and dns seems to be functioning as expected. This is a satellite link (viasat) so latency and transit times are long… I don’t know how to get the nameserver entry set since resolv.conf is a generated file and I don’t know where (up stream) to make a change. It is worth noting that this configuration with viasat is moderately new (a few months) and has been working until recently.
Thank you for the new logs. Yes, I can see that the Shake is showing a local network IP address too (a 192.168.x.x) in the eth0 parameter. This confirms that it is connected to the internet and that it can also see a time synchronisation NTP server that it can use.
I can see that, indeed, the Shake was transmitting to our servers until very recently. Do you know if, in the last days/weeks, something has changed in the local network where the Shake is connected? Maybe there is the need to add some exception (or its IP/MAC addresses) to the local firewall, since something similar has been solved in this way in the past.
The procedure to add other DNS, if you want to test it, is the following:
please open the dhcpcd.conf file with
sudo nano /etc/dhcpcd.conf
Delete all the lines that start with static, if present, thus leaving only interface eth0. If the line interface eth0 is not present, then add it. Do not modify anything else.
Then do:
sudo service dhcpcd restart
to make sure those changes took hold and then check the output file with
nano /etc/resolv.conf
This should renew the configuration, and possibly, solve the issue.
Thank you again for the update-
I’ve NAT’d 5555-5556 directly to the shake but that didn’t make a difference. Also checked NTP and it’s spot on (chrony can also get out to sync via NTP). Waited 5 minutes after a reboot and attaching logs. If you have anything else to check please let me know!
Ran tcpdump and I do not see any traffic going out on src or dst port 55555 or 55556.
No change except that the configuration is DHCP rather than a fixed IP as expected. UDP data is still arriving in San Jose. New log file attached.RSH.R5A78.2022-01-18T21_56_09.logs.tar (4.5 MB)
Thank you for the update Wang, and for the new logs.
It seems that new errors are starting to appear during the booting process, probably due to some corrupted files. I would, at this point, recommend a format and re-burn of the microSD card with a new copy of the Shake OS, to start with a fresh system and see if this solves the problem you are experiencing.
I will post the burning instructions here for your convenience:
Please take the microSD card you have and format it before burning the new Shake OS image
Make sure, when you format it, that the shown capacity is the maximum (i.e. if the SD card is 32GB like yours, then the capacity should be near or equal to that value). This is to check that no partitions have been involuntarily and erroneously created. They could be the cause of the error we see now
Take care to format the SD card in a FAT32 File System (or exFAT if the SD card is 64GB or larger)
I followed the numbered list points to burn the files on my SD cards, and I have not used Etcher or similar software.
If, even after this, the Shake is still displaying a “Not Connected” status, then please retry the following procedure to see if it solves the connectivity issue?
Please access your rs.local/ page, go to Settings (the gear icon high on the left), and then the Data tab. Try unchecking the Forward Data box and, after ~30 seconds, recheck it. Then,click save and restart.
I was able to identify that something happened on January 10th, when the Data Destination of our servers, raspberryshakedata disappeared from the configuration, while all other UDP destinations (as you confirm) are still up and running.
I honestly cannot see anything wrong with the Shake per-se. The boot procedure works fine, the NTP server is found, general internet is found, and you are able to see live data.
The only factor that is not as it should is the following:
Station Info
------------
Data-Sharing Mode : OFF
Data Server Conn : OFF
To try and solve this, could you please access the Shake rs.local/ page (if you are able to) and take a look, Settings page (the gear icon high on the left), Data tab, if the Forward Data box is checked?
Even if it is, can you please click on Save and Restart at the end of the page all the same?
If you cannot access the Shake rs.local/ page since this seems to be a remote installation, can you please ask someone who is able to to do the same process and see if the Shake regains connection to our servers?
One thought. When I went through the “Forward Data” dialog there was a pop-up that required accepting a license agreement. Perhaps that is “new” and the station has been on-line long enough that it was on-line before the license acceptance was part of enabling the “Forward Data” condition. Just a thought.
I finally had a moment to reload- New Pi, new SD card and only moved the geophone detector hat to the new PI. Rebooted a few times and let it sit for about 20 minutes, still not connected and I’m stumped
Thank you for taking the time of doing that procedure. I can see a different output now: the Shake finds the internet, finds a time service, and it is also set to upload data to our servers.
The problem seems to be focused on the data transmission ports. The logs show that:
2022 020 11:32:58>> Requesting connection to raspberryshakedata.com:55556
and that
2022 020 11:35:08>> Connection attempt #1 (raspberryshakedata.com:55556) failed with error code: Connection timed out
Could you please check again on your modem/router (for the IP address of the Shake) that the ports 55555 and 55556 are open in TCP and UDP traffic respectively? If they are not, could you please open them, wait for a while, and see if the Shake manages to connect?
If it doesn’t, then please repeat the previous procedure of turning off the Shake, turning of the modem/router, wait for a couple of minutes, turn on the modem/router again, wait that there is stable internet connectivity, and then turn on the Shake?
