GUI Connection Error

[usc]

Hello all, 

We've joined slightly late and just finished assembling the payloads last week.  Been running through some possible common issues that most of you have solved and could use some help on the RFD900.  The issue: "No acknowledge received, connection error" in the python GUI.

We've done all the steps in the resources for pi and RFD (configuring the RFD, setting up ad-hoc for pi, expanding SD card, connecting antennas, etc.)  The two RFDs actually maintain solid green lights and we get time stamps in the GUI, but when we attempt a connection test we get the connection error (and no pictures of course).  There are no other radio sources around the RFD's and they sit about 6ft apart from each other with the antennas pointing at the image payload.  

Are there any caveats or informal lessons learned that might push us in the right direction?  Thanks!  

[David MSGC]

Sounds like you have everything covered. Usually if you get this far and still 'No acknowledge" it is usually the program on the pi is not running, but you would not get the time stamp if this were the case.
Does it give a time stamp when you do a connection test?
You can watch the LED on the camera and each time it takes a picture the LED will light, if it lights after watching it for at least 60 seconds you know the program on the pi is running. IF you watch it for at least 60 seconds and you don't see the LED light the program on the pi is not running. You don't have to worry about ad-hoc being connected since this is only used for the laptop to ssh over wifi with the payload and is not used for the RFD communication.
A few things I have seen even when I thought everything was proper is sometimes you have to get the radios farther apart, like 10 feet or more, I have had them act wierd when sitting on the bench within a few feet of each other. I have also seen McAfee or security software block the serial port, sometimes you have to add a firewall exception to putty or python from the windows firewall settings, if Mcaffee is controlling the firewall it usually works okay, but sometimes after an update it will quick working and windows firewall is blocking the serial port. Also try using a different USB port on the laptop, I have had this make a difference too.
Expanding the SD card and everything should have been done already, if anyone has gone into raspi-config make sure that ssh over serial has not been enabled. When disabling ssh over serial it will give a message similar to "disabling serial" and you don't want to disable serial, but it is disabling ssh over serial. Type: sudo raspi-config and it will open up the config utility and somewhere you will see "enable ssh over serial" you want to say no to this and disable ssh over serial. Reboot and try again. I have had the same problem where I can get an initial time stamp and connection and nothing afterward and it was ssh over serial being enabled. This adds encryption data to each serial packet which we do not need nor does the program know what to do with the extra data, it is only expecting one byte/one character.
I would first try to separate the radios farther from each other and make sure ssh over serial is disabled and try again.

[usc]

Does it give a time stamp when you do a connection test?

        I get time stamps in anaconda and the GUI until I do a connection test, to which I then receive a connection error.

You can watch the LED on the camera and each time it takes a picture the LED will light, if it lights after watching it for at least 60 seconds you know the program on the pi is running. IF you watch it for at least 60 seconds and you don't see the LED light the program on the pi is not running.

        We are getting the LED on the camera.

Disabling the SSH over serial actually worked in sending and receiving the image request (thank you!), to which a file was saved as "image_20170419_###" on the GS laptop.  The issue now is that the image file sent is empty; 0 kb.  It is exactly the same issue and screenshot as member deniseund in #Raspberry-Pi-failing-to-send-photo-packets.  We checked the RFD connection using 3DRRadio and connection is nominal.  Just some more digging/troubleshooting; will post an update with any findings.

[David MSGC]

Yes you should get a time stamp with each connection test, if the connection test fails it will not give a time stamp and will give an error message. The image file name should be image_00x, and x will increment with each picture taken. If you get a filename like image_gobbldygook059lakf003kl the radios and/or program is out of sync. Your symptoms still seam like ssh over serial is enabled. When you send a request to the payload the tx led on the ground radio and the rx led on the payload radio should blink once for a very short period of time. When the pi radio responds back to the ground station side the pi side radio tx led and the ground side rx led should only blink once very quickly. If the pi side radios tx led blinks more than once and the ground side radios rx led blinks more than once than ssh over serial is enabled or the settings on the radios are not set right. ECC, CTS/RTS, and mavlink should all be off on both radios settings. The pi and ground station are only sending one byte back and forth during a request and acknowledge. This is the fast one blink of the led. If there are any error correction, rts/cts, mavlink, or ssh enabled it will send several bytes and the firmware is only expecting one byte.
The signal check in 3dr should not indicate a square wave on the remote radio plot, if it is it will give results similar to what you are seeing. If this is the case power cycle the ground side radio by unplugging and reconnecting the connector right at the radio, not the usb at the computer. If you disconnect from the computer you will have to restart the program on the ground station side.

[irowe]

Apr 17, 2017 17:52:37 GMT -7 David MSGC said:

You can watch the LED on the camera and each time it takes a picture the LED will light, if it lights after watching it for at least 60 seconds you know the program on the pi is running. IF you watch it for at least 60 seconds and you don't see the LED light the program on the pi is not running. 

If I don't see the light on the camera but my RFDs are connected, then how can I get the program to run on the pi? I assume this involves connecting to the laptop via an ad-hoc network, but what exactly do I change to make sure the pi runs the still image program?

[David MSGC]

If the program on the pi is not running than the crontab settings have been changed or the camera cable is not connected properly. Also make sure you have the SD card for RFD and not ubiquity. The ubiquity SD card will not have the RFD program start at startup or reboot. Crontab is a scheduling program for Linux used to start programs and processes on a repeating schedule. (in this case at reboot or start)
If the crontab settings have not been changed than make sure the camera cable is inserted completely and the proper orientation. (there are pictures in the RFD instructions/slideshow from the workshop material)
If someone has changed the crontab settings you will have to connect to the pi over ad-hoc or use a keyboard and monitor. Enter "sudo crontab -e" at the command prompt after logging in. It will open a file in the text editor for the crontab. After all the notes at the top of the page you will see a line that starts out with, @reboot cd /home/pi/RFD_Payload; sudo python /home/pi/RFD_Payload/RFD.... Make sure there is no # at the beginning of this line. If there is not a # at the beginning of the line the RFD program will run whenever the pi is started or rebooted. If there is a # at the start of the line make sure you have the RFD SD card and not Ubiquity. If you do have to change the crontab file make sure to press ctl-o to "write out"/save your changes and ctl-x to exit the editor.

[irowe]

David MSGC Thanks for the instructions on checking the Pi code

Looks like everything is as it should be with the chrontab. I do notice, however, that in the RFD_payload folder there is both a RFD_python_Pi.py and a Modified_RFD_pythong_Pi.py. Chrontab is running the non-modified one.

The more worrying thing is that I see the LED on the camera light up once at start-up, and then no more after that. I watched it for a couple of minutes and it never came back on. Tried shutting down the Pi via SSH (sudo shutdown -h now), powering off, and then powering on again, but I do not see the camera LED after the boot. The RFDs are connected just fine, but I never get a connection with the Still Image python script ("No Acknowledge Received, Connection Error"). The camera looks like it is connected properly.

I'm assuming the problem lies with the Pi, but does anyone know what it could be?

Edit: Looks now it works fine. I think rebooting did the trick. I also had to disconnect from the pi's ad-hoc network.

[psuteam]

Hi. I was having a similar problem and rebooting worked for me as well. However, when doing multiple tests in a row, I've found that the connection is very inconsistent. Sometimes it'll work as expected, sometimes I'll have to reboot the computer again, sometimes I'll have to unplug and shut down everything and start from scratch.
The payload is set at a good distance in front of the dish and the green light on both modems are fairly consistently solid.
Do you have any idea why the system is being so inconsistent?
And if you think its still the disabled ssh over serial issue, can you please explain that in more detail? Thanks!

[David MSGC]

If ssh over serial is enabled the still image transmission will never work, it will not be inconsistent. Being connected to ad-hoc or not will not affect the transmission of pictures.
Are you using the system indoors? If you are inside a building you will have to point the antennas away from each other, get the antennas off the ground or point them away from the ground. The only time I have had any inconsistent behavior is while indoors with the antennas close to each other, facing each other, too close to the ground, or too much 900MHz noise. You can check noise levels from the 3dr program in the rfd_resources folder. Too close to the ground can always be a problem, whether you are indoors or not, get if off the ground or lay the payload on its side.
You can remedy all of this by lowering the transmit power during testing, but I do not recommend this since there is a chance someone will forget to raise the transmit power before a flight and you will lose connection and not be able to increase the transmit power during flight.

can you also provide more info on what it is or is not doing when it is inconsistent? Is it getting one packet of the picture, no packets, are there errors in the console, etc.
If you receive a few packets and drop some, receive a few then drop some this is usually too much noise or you are indoors with the the antennas facing each other or too close to each other. If you cannot begin to receive a picture and it fails the first packet and times out it is usually the radios not being paired or config settings incorrect. If you can receive at least one packet the config/settings are correct. You should never have to change any settings like ssh over serial or anything. If you have received at least one packet in the past and have never changed any settings than all the settings should be left alone, there is no reason to be changing radio or raspi-config settings.
If all the settings are correct, you have received at least one packet in the past, and it is not working: 99% of the time it is the radio not being paired, the radios are too close and interfering with each other, or there is too much 900MHz noise.

[psuteam]

Either it will receive images just fine right away after start up or not at all. I did a test outdoors recently with the payload nearly 100ft in front of the dish. This is how it went:

When first opening the GUI, Anaconda started saying "Waiting for Acknowledgement". After giving it a few seconds with no change, I unplugged RFD from the laptop and restarted the laptop, tried again...same problem happened. So then I unplugged RFD, turned off payload, unplugged batteries, re-connected everything and tried again... Anaconda then immediately responded with a ping time when the GUI opened and images were transmitted just fine.



Is it possible that operating the Still Image system incorrectly in the past may have corrupted something? Sometimes RFD would be plugged in without the payload being on and for a while the payload was being turned off without getting shut down in PuTTY.



Thanks!

[David MSGC]

IF the SD card is corrupt it would not be intermittent, it would just not work. Usually the pi will not even boot if the card is corrupt.
Are you waiting for the acknowledgement to time out and the GUI to open or are you closing it before the GUI opens?
The payload will not be able to acknowledge when first starting up or taking a picture. The RFD payload needs to be on for at least 65-70 seconds before starting the program on the ground station computer. The payload Pi takes about 60 seconds to start up and the RFD program to start running, it then takes a picture right away which takes a little more than 5 seconds. During this time it cannot acknowledge the ground station, the RFD payload has to be on for at least 65-70 seconds before you can start the ground station program. If the GUI times out waiting for an acknowledge and the GUI opens with a connection error you will never be able to download an image, the programs are not in sync or talking to each other, if this happens just close and restart the program to try to establish connection. The only time you should ever have to unplug the ground side RFD radio is if the green LED will not stay solid, unplug it from the radio not the computer. It does not matter if the payload is on and the ground station is not. It does not matter if the ground side radio is plugged in without the program running. All the payload side program is doing is taking pictures every 60 seconds and listening to the serial port for commands in between pictures, if it is taking a picture it will not be able to acknowledge a command, if the program has been running and connected it will acknowledge once the picture is done being taken. If the payload is on and the ground radio is plugged in it does not matter if the program is running or not, it is preferable to have the radio plugged into the computer for a bit before the program is started just make sure the LED does not blink and you know the radios are paired, then start the program on the ground station computer. If the gui ever times out waiting for an acknowledge at start up or seems to not respond, close and restart the ground station program. You should not have to restart or power cycle the payload at all. This is by design, since you will not be able to power cycle once the balloon is in the air. The ground side program can get stuck in a loop waiting for an acknowledge from the payload, it does this on the ground side and not on the payload side because you have access to the ground station computer during flight and not the payload so you can just close and reopen the program on the ground side. The only time you should have to unplug the RFD cable from the computer is if you get com port errors on the ground station. If you ever feel the system is not working right always leave the payload on, there is no benefit to power cycling the payload, just close the GUI and restart the program on the ground side. You should not have to unplug the RFD radio unless the LED is not staying solid or you get a com port error(this is assuming you have the right com port set in the program, and the com port will not change unless you switch FTDI cables or computers)
Turn on the payload first(does not matter if RFD is plugged into computer and does not matter if computer is on or not), within a minute after the payload is powered on you should see the LED on the camera light for a few seconds, this means it is taking a picture and you know the program is running. If the computer is not on yet turn it on. If you have not plugged the RFD to the computer yet plug it in. Watch for the green LED on the RFD radio to stay steady, if it blinks even once over a period of 30 -40 seconds the radios are not paired, power cycle the radio by unplugging from the cable not the computer. Make sure at least 60 - 70 seconds have passed before starting the RFD program on the computer. Start the program, if it says waiting for acknowledge let it keep trying until it gives up, if it gives up close the GUI and restart the program. If none of these steps work you are too close to the payload or the antennas are too close to the ground.

[psuteam]

Thank you, that helped to clear up a lot of confusions!