A beautiful day at Skalanes
by on with 1 Comment

Today was another sunny, beautiful day at Skalanes. We woke up to the sun shining down and just a couple of clouds in the sky. We started the day with breakfast outside on the patio and then we all got to work.

There was a lot that was being worked on today. Erin and Nic were working on flying the drone. Erin was controlling the drone outside while Nic was in the door working on the computer. A Nexus running the Dji application is placed in the drone controller for flying, but the Nexus also needs to be connected to WiFi to communicate with the ground station program on Nic’s computer. Once they stepped outside of the room, the Nexus was losing WiFi access. We found really long micro USB cable so the Nexus could stay inside while the controller was outside with Erin flying blind. The ground station program on Nic’s computer allows the drone to go on autopilot. They were eventually able to get it to go on autopilot for a few minutes which was pretty cool. It was a little freaky yesterday when we told the drone to come home (where it took off from), then Nic decided to land it somewhere else so he tried to take control but the drone wouldn’t release control of itself. Nic was able to make a complete flight plan on the ground station software for the archaeological sites. I’ve posted a couple of pictures of Erin flying the drone and the final flight plan below.

Tara did a lot of work on the soil samples. Instead of putting them in the oven, she took advantage of all the sunshine and had them dry in the sun for the past two days. They need to be dry in order to work with the Nano IR for spectroscopy. Tara sifted soil samples to remove the big rocks and then crushed what was left to make the samples more sandy. The standards that Tara made back in Richmond for the soil samples here were made with sandy soil, which is what much of Iceland is. The 20 samples from the glacier were much harder than the tephra layers from the riverbed. The tephra layers were basically mud that dried so they didn’t need to be sifted, just crushed to make sure they had the consistency of sand. This was all done outside on the deck and it was sunny enough that she got sunburnt! Something to think about next time is bringing a mortar and pestle to crush the soil. She was using some kind of fishing hard plastic thing to crush, which worked out surprisingly well. There were a couple of samples that were too rocky so she took some from the soil bag and let it dry in the sun that day too. Good thing we took more soil than we needed!

I did some work on Field Day and the Ambiance platforms sketches. We noticed that the altitude on from the sensors on the Ambiance platform was a little off — when we were upstairs in our ‘office’ it was telling us we were -40 below sea level. Charlie made a calibration code for the devices where it takes a known altitude that the user provides through Field Day and makes a correction value that is applied to the altitude taken from the sensors. So if we go down to the sea and give a calibration value of 0 since we’re at sea level, and the sensor is giving us -4, the correction value will be +4. The sensor isn’t really stable enough to give us exactly zero for a reading, but it’s pretty close.

I also did some work on the BLE reliability with Field Day. I left the platform running with Field Day plugged into my computer so I could look at the logs when it eventually stopped. I realized that after a little while the messages to Field Day would eventually get jumbled which caused Field Day to be confused. Field Day was sending messages too often, so it hadn’t finished processing the response before the next response came. This was done using a Handler and Runnable. I changed Field Day to only send another request to the bluetooth device once the response from the previous request had been processed. This has helped a lot. Before Field Day was only getting a hour or so of good reading from the device, but I left it running last night and it went from about 7:30 at night until 2 in the morning. I was really happy with that result. I learned that native Android has known issues with Bluetooth Low Energy and different Bluetooth chips has different issues as well. One thing that stuck out when I was reading was the unreliability with sending and receiving longer messages. The Ambiance sketches send long messages because there are so many sensors on the platform.

Deeksha worked on the visualization and produced some CSV’s from our trip on Heimaey and the glacier for Erin to put into GIS which was pretty cool. Charlie worked on added the BME280 chip to Dart, one of our ambiance platforms and rewiring Grace to work with the ambiance sketch pin settings. Gail worked on thinking about this trip in terms of wilderness and outdoor education program. She’s been thinking about sample day plans, how long we would need to stay at places, what kinds of topics would be interesting where, and what they would need to bring or have available. Needless to say, it was quite a long and successful day at Skalanes.

We finished the day with a bonfire at the beach. Our group, Oli, Rannveig and the Glasgow group that’s also staying at Skalanes were there enjoying the sunshine and hard work of the day. Erin and Charlie jumped all the way the water which was ice cold! Gail, Tara and I just stuck our toes in which was enough since the water so cold it hurt!

 

 

Lab Notebook on Field Day
by on with No Comments

This week, I’ve done a lot of work on Field Day. It doesn’t seem like much but that’s because I kept getting caught up in tiny errors during coding. The biggest thing that I’ve added to Field Day is part of the Lab Notebook activity. We decided this week that integrating Google Drive is less important than being able to open local files. We decided this because we will not always have internet access in Iceland and that’s essential for Drive if we want to update our files. The Google Drive API for Android also has limited options. The Android API only allows the application to access files that were created by the application itself. Since most of our files are PDFs and the Android API doesn’t allow uploading, we decided to move away from that for now. There’s a Java Client Library for Google Drive which has many more capabilities that we will probably use when we go back to Google Drive.

I’ve begun working on the local documents viewer and have successfully downloading files and displayed the pdfs. There’s now an option to download files where the user provides the URL of the server where the files are and the path to the directory with all of the files in it. The code relies on the directory having indexing enabled. Field Day goes out and queries the directory and returns a list of files and folders. The user is asked to select which files to download. If a folder is selected Field Day goes out and queries that folder, etc. until the user hits ‘Download.’ The files are downloaded and stored on the external storage (the SD card) keeping the directory structure that Field Day saw in the remote directory. A list of files of is displayed on the screen to the user. Upon clicking a file, the file is displayed. Currently it works for only PDFs and JPEGs which are handled as Bitmaps. Text files are not. If a folder is clicked then the folder is opened and the files and/or folders in that folder are displayed.

Sending data back and forth
by on with No Comments

As you remember, last week I finished the new data model implementation within Field Day and downloading remote database data. Field Day writes sensor database to the local database respecting the new model — populating fieldday_streaming, fieldday_reading, and fieldday_spot.

The big hurdle that still needed to be hopped over was sending those newly populated tables back to the remote database. This week I got over that big hurdle. The user can now upload the local tables to the remote tables. Field Day gets all of the rows in the tables I mentioned above, starting with fieldday_spot because the other two have foreign keys that reference the spot table. Moving the rows from the reading and streaming table is simple, but moving the spot table has different characteristics. If people are pushing to the database at the same time, we have to make sure that we are only inserting one of each ‘trip, site, sector, and spot’ which make up the primary key in the database. Field Day takes the ‘tripID, siteID, sectorID, and spotID’ of each row in the local database and queries the remote database asking for the count where a line has those four columns that equal the values in the local table. If the count comes back zero, then Field Day inserts the row. Each tables updating is wrapped up in a transaction, so if something goes wrong Field Day rolls back the transaction and sets the database to it’s previous state. Once the inserting has successfully finished, the database in the local table is copied to external storage in an /archive directory and the tables are dropped and created again so they are empty.

I moved ‘Take a Sample’ on the main screen to ‘Sampling’ and created a new option under that menu for ‘Database Actions.’ The downloading from remote database activity was previously under the Settings option, but since the application is basically doing the same thing in each, I merged them in to one activity. There are three buttons for the user to choose from (which you can see in the picture below) — setup local database, upload tables, clear local database. Upon the press of any of the buttons, the user is asked to confirm.  When the user selects clear local database, just like after the user upload the tables, the local database is copies and wiped.

A couple other small things that I’ve worked on this week are integrating the Google Drive Android API and fixing small things with the sensor sampling activity. As for the Google Drive API, I’ve gotten Field Day to the point where it asks for permission to access Google Drive, but doesn’t do anything after that. To my knowledge, this API only allows the app to interact with files that were created by the application itself. I’m not 100% sure of that so I’m still working on it. Small things that are fixed are the user is notified if they haven’t set up their local database with a remote database, which means they can’t write anything to the local database. When a Spot number is put in the spot edit text on the sampling activity, Field Day checks to make sure that spotID doesn’t already exist in the local database if it does, then it doesn’t write to the database.

 

device-2016-05-19-094627

Field Day had some internal work done
by on with No Comments

If you’ve read my last post, you know that Field Day got a external makeover. It also got a lot of work done internally. For the past couple of weeks, Deeksha, Charlie and I have been working on the new data model for Field Day and the PostgreSQL database on hopper (or our portable machine). We’ve finally figured out the first pass. As you know, this could change as we move forward and test our designs, but this has gone through many field practices. There’s more than one table now — there’s pretty much one for each data point we are collecting (sensor, platform, host, site, sector, etc) all of which have unique keys and all of which will be available and populated before we use it in the field. This week, I integrated that into Field Day.

We decided that we wanted Field Day to connect to a remote database and retrieve all of the table data and store it into a local database. Field Day populates sites and sector drop downs in the ‘Take a Sample’ dropdown from which the user can choose the site and sector they want at that stop. Field Day can also check whether or not the platform and sensor they are using are in the database. Currently, Field Day fails if it can’t find the platform, but I’ll be making this more robust this week. Allowing the user to retrieve external database data will prevent inconsistencies with spelling, capitalization, human error, and the like which are problems we’ve had in the past.

New design!
by on with No Comments

This past week has been busy! I’ve done a ton. One of the things I’ve done is make Field Day look a little prettier. I find it much easier and productive and effective to work on if the application is prettier and nice to look at. Go back to my previous posts and you can see that Field Day was ok looking, but it was really dark and boring. Now it’s quite pretty and fun. I got rid of the circles and just made it icons without a background. I also made it look like the user is in a field, hence ‘Field Day!’ There are probably some other things that I could add, but for now I think it’s pretty and much nicer to work with. You can see pictures of it below.

 

fieldday-main fieldday-sample

Ambiance and a Feather
by on with No Comments

So, it’s been a little while since my last post. Here’s a quick update. I assembled our ambiance platform using the Adafruit feather and two sensors: Altimeter and Temperature, Humidity and Pressure. The Adafruit feather is a tiny BLE capable micro controller (like a Light Blue Bean). We decided to go with the feather because the BLE chip it uses is Nordic, like the Red Bear Labs BLE Shield. With the Nordic chip we can make our own services with UUIDs. The Bean has a built-in set of UUIDs and communicates differently than the Nordic chip.

The platform sensors currently use two different communication protocols: I2C and SPI, but we decided that probably wasn’t going to be a problem. I was able to solder on the sensors to a prototyping board and upload a sketch. I tested without the BLE first time to make sure it was working. It was, thanks to sample code from Adafruit! Adafruit has a different BLE library but it’s all the same principal. I tested the BLE code with Field Day and all works well. A picture of the ambiance platform is below.

Light Blue Bean!
by on with No Comments

In my last post, I said that I was going to switch all of our individual fragments of sensors to one that is ‘Bluetooth Sensors.’ Since then, I have finished that and redesigned even more of Field Day.

After I finished switching all the fragments to one, I dove into working on BLE connection with the Light Blue Bean (LBB). Unfortunately, the LBB does not do Bluetooth the same way the Red Bear Labs shield does. The LBB uses something called ‘Scratch Characteristics.’ These are built-in 5 characteristics that can be written and read from the client (in this case, Field Day) and the server (the LBB). These characteristics have set UUIDs, so that means I can’t use the custom UUIDs I set for the RBL’s sensor. After browsing some of the android SDK code from Punchthrough (the people that made the LBB), I was able to determine the UUIDs that are used for the characteristics. Since Field Day is going to have to determine what kind of device it is connected to, I redesigned it so that the fragment is cleaner and doesn’t do any of that work. There are separate classes for a Bluetooth Sensor, GATT Client, and Bluetooth Service. The Service talks to the Sensor and the Sensor talks to the GATT client and determines what type of device it is connected to by checking the UUIDs. All the Fragment does now is say when it wants to write a message or read a message.

Punchthrough has sample code available for arduino and reading and writing the scratch characteristics. The examples along with the SDK has proven helpful when rewriting the android code to accommodate for the Bean. Unfortunately, not many people have used the Bean with Android. All of the examples are with iOS. Field Day is able to read scratch characteristics just fine from the Bean, but is currently unable to write to any characteristics. One huge problem with the bean is that everything is Bluetooth. It’s not like arduino where you plug in the device to your computer to upload code and power it on. This means that the Bean can only be connected to one thing at a time and thus, is really hard to debug. Typically when debugging arduino devices, I’m able to plug the device into my computer and watch the Serial monitor for debugging code I’ve added in. With the Bean, I cannot. It’s blind debugging and it sucks. I’m still working on figuring out how to write a characteristic and have the bean read that characteristic, but I think I’m getting closer. I hope to finish that this week.

BLE in Arduino and Android
by on with No Comments

I have accomplished quite a bit within the past week, if I do say so myself. Last week, I said that I had begun working on the BLE part of Field Day. Well, after some struggle I finished it! I am able to send a request to the Arduino device and upon receiving that request, the Arduino device sends a message back to the android device and writes to he screen (woo!). But the code is not pretty. Android provides a Bluetooth Low Service example in Android Studio, but it uses deprecated code. I researched and was able to modernize it.

Bluetooth Low Energy is pretty complicated as a service. It uses something called GATT (Generic Attribute Profile). The bluetooth server (arduino device) has a GATT profile and within that GATT profile there are services and services have characteristics. Each service can have multiple characteristics, but there can only be one TX and one RX (read and write) characteristic per service. I learned this the hard way. You can see a diagram of what I’m talking about below.

For the arduino side, we use Red Bear Labs BLE shield. There is a standard BLE library for the arduino but it’s really complex. I’ve read over the code multiple times and I’m just now grasping a little bit of it. RBL has constructed a wrapper for that code. They broke it down into 10 or so different functions, which I must say is mighty useful. I used that when constructing the Arduino code.

During my coding of the BLE on android I discovered that most of our Fragments of sensors are all the same except for the names of them. After talking with Charlie we’ve decided that we’re going to get rid of the individual fragments for each sensors and just have one that is ‘Bluetooth Sensors.’ I’m working on moving the BLE code to that setup now. Hopefully I will be done with it within the day.

 

GATT

Handlers handling handles
by on with 1 Comment

After some testing of the Handler implementation in Field Day, I’ve determined that it is actually working! I was finally able to look at the data that I had been writing to the database.

Databases in Android are stored in internal (private) memory. There are Android apps that let you look at the files on your device, but only for external storage. The internal storage is private. What I ended up doing was adding a function to write the database from internal storage to external storage. Once it was on external storage, I used a SQLite Viewer Android app to look at the database and it looks good!

I did some research into the application state when the device screen is turned off. In our old application, Seshat, the activities would die when the screen was off. That’s not good. We want to be able to turn off the screen and hang the Android device from our person some how. It’s useful to have our hands free. After a couple hours of research, I determined the libraries we want to look at are in the PowerManager library on Android, particularly the PARTIAL_WAKE_LOCK state. I wrap the methods I want to happen when the screen is off in what are essentially ‘start’ and ‘stop’ methods. The methods were not executing with the screen off, so I researched some more. I think what will ultimately happen is switching the SensorSampleActivity to a Service. A Service in Android is one of the top priority processes. It’s one of the last to die to if the device needs more memory. It can continue running even if the Activity the started it has died. Since the Activity is now working, I’m going to hold off in converting it to a Service.

The last thing I worked on this week was Bluetooth LE in Android. Tara gave me a BLE shield with an Arduino attached to test my code with those devices. Android provides a BLE example in Android Studio so I’ve been working with that. However, the example they provide has deprecated code so I’m working on modifying it to the new setup. It’s not working yet, but I’ve only been working on it for a couple of hours!

sqlite-fieldday
Sample of sqlite database on Field Day
1 2 3 4