Long Day’s Journey Unto Ice
by on with No Comments

Today began at 7:30 when Charlie and Emi left to pick up the rental van at the airport in Egilsstaðir, roughly 30-45 minutes away. Sanne, who runs and maintains the house at Skalanes, drove them – and as much luggage as we could pack in – out to the parking area where we had one rental car waiting. They transferred people and luggage into the rental and drove off. The rest of us ate breakfast, cleaned up the house as best we could to help out Sanne and the others who run Skalanes (a group of 30 is arriving at Skalanes tomorrow!), and began to load up Sanne’s vehicle with all of the luggage we had not been able to send with Charlie and Emi. Once the luggage was loaded, we began walking out the drive while Sanne drove the luggage to the parking area and then turned around to pick us up and drive us the rest of the way. At the parking area, we were faced with a cold and windy wait as Charlie and Emi drove back. We amused ourselves with word games and hypothermia dances until finally they arrived.

Loading up the vehicles with luggage and humans took only a few minutes, and then we were off on our long drive back to Vik. We stopped for a few minutes at Glacier Lagoon so Marketing could get some nice picturesque footage of us in Iceland and briefly saw a seal – they occasionally swim up into the lagoon, which is apparently close to 200 meters deep due to the icebergs. Other than that, the drive was long and pretty uneventful, with stops only for restrooms and other necessities. At one of the gas stations we learned that today is a holiday here in Iceland (Whit Sunday, the day before Pentecost), and we were concerned that the grocery store here in Vik would be closed, preventing us from getting groceries for tomorrow. We made a short detour to Klaustur, where we knew there was a gas station/convenience store that would have some basic groceries available, and picked up materials to make some quick Mac and Cheese for dinner tonight and sandwich materials for lunch tomorrow.

When we pulled into Vik we discovered that the grocery store is in fact open today and we needn’t have worried, but the early shopping trip did allow us to begin working on dinner much sooner after we arrived. Everyone was tired and hungry, since we pulled into the hostel at 7 pm and had left Skalanes at 8:30 (or earlier) this morning. It’s after dinner now, and Charlie and Neil are working on getting the elevation platforms and drone ready for tomorrow, when we’ll be returning to the Sólheimajökull glacier. Rather than collecting soil samples like last time, we’ll be climbing on the glacier itself to collect geographical and volumetric data we’ll use to chart its recession over time. Nic is down with a nasty cluster headache, which is preventing him from helping get the drone ready. Yesterday I finished modifying and debugging the elevation platforms’ code, but now Charlie is trying to debug an issue with Field Day, the Android app we use to gather data from them. All in all, it’s a pretty regular night for us here, with a group playing Euchre in one area and tech gear exploded across another space.

Tomorrow we leave the Puffin Hostel at 9 am to meet our glacier guides at 10 and begin collecting data on the ice.

Catching-up
by on with No Comments

In some sense the amount of activity going on is inversely proportional to the posts we’re making here to document it… Here’s a quick summary of what we’ve been working on since my last post:

o LiDAR – We have new unit which is much more capable. We built a test-rig in Hopper to simulate having it on Kia, we’re just starting to collect data. Nic and Kellan are working on the algorithms/workflow to analyze it. Charlie is working on mounting it on Kia.
o Image processing – Kellan and Nic are working on a workflow for extracting features from the images. Charlie built a map (headed to QGIS) with marked known POIs on the Skalanes peninsula to use as part of the training data.
o Glacier forefield sampling – Tara and Charlie are developing a sampling plan and 16S RRNA workflow for processing the soil samples.
o Benchmarks for registering multi-modal data collected by sensors on the RPA.
o Discovered Siggi’s yoghurt.
o Kristin and Charlie are working on Field Day, mostly on the BLE plumbing.
o Checklist App – More news to follow.
o Gail and Charlie have done lots of logistics and planning.
o Emi is working on the avian surveys and the MinION workflow for processing glacier forefield soil samples
o New ambiance platform (single chip) is designed, construction to follow.

More regular updates to follow, we’re all psyched.

Some set backs
by on with No Comments

This week has been a bit frustrating, but it all seems to be looking up now.

After taking some test images with the UAV last weekend I realised that there was image fragmenting going on with the camera. This meant that one of the cables was either lose, dirty, or damaged. So, after quite a few hours of hardware debugging (cleaning plus some cable re-seating, etc) Im reasonable certain the ribbon cable that transfers the actual image to the tablet receiver needs to be replaced. I still have no idea what caused the actual damage in the first place, but thats a little beside the point. Unfortunately that means until that comes in the camera will be out of order. (…Charlie ordered two just in case…)

BUT FORTUNATELY, that gave me an excuse to take the whole camera gimbal off which means its time to fly without the camera and stress-test how much of a payload it can carry! *cough* I mean, purely in the interest of knowing our payload total for the UAV+CAMERA+LIDAR.

In other news, after setting up a physical router I was able to solve the uplink problem. Essentially it turns out that having complete control over your ports when trying to control ~6 different java servers worth of information transferring all at the same time is really useful.

Lab Notebook on Field Day
by on with No Comments

This week, I’ve done a lot of work on Field Day. It doesn’t seem like much but that’s because I kept getting caught up in tiny errors during coding. The biggest thing that I’ve added to Field Day is part of the Lab Notebook activity. We decided this week that integrating Google Drive is less important than being able to open local files. We decided this because we will not always have internet access in Iceland and that’s essential for Drive if we want to update our files. The Google Drive API for Android also has limited options. The Android API only allows the application to access files that were created by the application itself. Since most of our files are PDFs and the Android API doesn’t allow uploading, we decided to move away from that for now. There’s a Java Client Library for Google Drive which has many more capabilities that we will probably use when we go back to Google Drive.

I’ve begun working on the local documents viewer and have successfully downloading files and displayed the pdfs. There’s now an option to download files where the user provides the URL of the server where the files are and the path to the directory with all of the files in it. The code relies on the directory having indexing enabled. Field Day goes out and queries the directory and returns a list of files and folders. The user is asked to select which files to download. If a folder is selected Field Day goes out and queries that folder, etc. until the user hits ‘Download.’ The files are downloaded and stored on the external storage (the SD card) keeping the directory structure that Field Day saw in the remote directory. A list of files of is displayed on the screen to the user. Upon clicking a file, the file is displayed. Currently it works for only PDFs and JPEGs which are handled as Bitmaps. Text files are not. If a folder is clicked then the folder is opened and the files and/or folders in that folder are displayed.

Sending data back and forth
by on with No Comments

As you remember, last week I finished the new data model implementation within Field Day and downloading remote database data. Field Day writes sensor database to the local database respecting the new model — populating fieldday_streaming, fieldday_reading, and fieldday_spot.

The big hurdle that still needed to be hopped over was sending those newly populated tables back to the remote database. This week I got over that big hurdle. The user can now upload the local tables to the remote tables. Field Day gets all of the rows in the tables I mentioned above, starting with fieldday_spot because the other two have foreign keys that reference the spot table. Moving the rows from the reading and streaming table is simple, but moving the spot table has different characteristics. If people are pushing to the database at the same time, we have to make sure that we are only inserting one of each ‘trip, site, sector, and spot’ which make up the primary key in the database. Field Day takes the ‘tripID, siteID, sectorID, and spotID’ of each row in the local database and queries the remote database asking for the count where a line has those four columns that equal the values in the local table. If the count comes back zero, then Field Day inserts the row. Each tables updating is wrapped up in a transaction, so if something goes wrong Field Day rolls back the transaction and sets the database to it’s previous state. Once the inserting has successfully finished, the database in the local table is copied to external storage in an /archive directory and the tables are dropped and created again so they are empty.

I moved ‘Take a Sample’ on the main screen to ‘Sampling’ and created a new option under that menu for ‘Database Actions.’ The downloading from remote database activity was previously under the Settings option, but since the application is basically doing the same thing in each, I merged them in to one activity. There are three buttons for the user to choose from (which you can see in the picture below) — setup local database, upload tables, clear local database. Upon the press of any of the buttons, the user is asked to confirm.  When the user selects clear local database, just like after the user upload the tables, the local database is copies and wiped.

A couple other small things that I’ve worked on this week are integrating the Google Drive Android API and fixing small things with the sensor sampling activity. As for the Google Drive API, I’ve gotten Field Day to the point where it asks for permission to access Google Drive, but doesn’t do anything after that. To my knowledge, this API only allows the app to interact with files that were created by the application itself. I’m not 100% sure of that so I’m still working on it. Small things that are fixed are the user is notified if they haven’t set up their local database with a remote database, which means they can’t write anything to the local database. When a Spot number is put in the spot edit text on the sampling activity, Field Day checks to make sure that spotID doesn’t already exist in the local database if it does, then it doesn’t write to the database.

 

device-2016-05-19-094627

Field Day had some internal work done
by on with No Comments

If you’ve read my last post, you know that Field Day got a external makeover. It also got a lot of work done internally. For the past couple of weeks, Deeksha, Charlie and I have been working on the new data model for Field Day and the PostgreSQL database on hopper (or our portable machine). We’ve finally figured out the first pass. As you know, this could change as we move forward and test our designs, but this has gone through many field practices. There’s more than one table now — there’s pretty much one for each data point we are collecting (sensor, platform, host, site, sector, etc) all of which have unique keys and all of which will be available and populated before we use it in the field. This week, I integrated that into Field Day.

We decided that we wanted Field Day to connect to a remote database and retrieve all of the table data and store it into a local database. Field Day populates sites and sector drop downs in the ‘Take a Sample’ dropdown from which the user can choose the site and sector they want at that stop. Field Day can also check whether or not the platform and sensor they are using are in the database. Currently, Field Day fails if it can’t find the platform, but I’ll be making this more robust this week. Allowing the user to retrieve external database data will prevent inconsistencies with spelling, capitalization, human error, and the like which are problems we’ve had in the past.

New design!
by on with No Comments

This past week has been busy! I’ve done a ton. One of the things I’ve done is make Field Day look a little prettier. I find it much easier and productive and effective to work on if the application is prettier and nice to look at. Go back to my previous posts and you can see that Field Day was ok looking, but it was really dark and boring. Now it’s quite pretty and fun. I got rid of the circles and just made it icons without a background. I also made it look like the user is in a field, hence ‘Field Day!’ There are probably some other things that I could add, but for now I think it’s pretty and much nicer to work with. You can see pictures of it below.

 

fieldday-main fieldday-sample

Scheming…
by on with 1 Comment

After a week or so of tinkering we have a new version of the database schema that supports Field Day and the visualization stack. Here is the fancy entity relationship diagram for it:

IMAG0420

Field Day will load trip, site, and sector information from Postgres (config option) and cache them on the tablet. Deeksha is committed to moving the data from Iceland13 & 14 and Nicaragua14 into this new structure.

Light Blue Bean!
by on with No Comments

In my last post, I said that I was going to switch all of our individual fragments of sensors to one that is ‘Bluetooth Sensors.’ Since then, I have finished that and redesigned even more of Field Day.

After I finished switching all the fragments to one, I dove into working on BLE connection with the Light Blue Bean (LBB). Unfortunately, the LBB does not do Bluetooth the same way the Red Bear Labs shield does. The LBB uses something called ‘Scratch Characteristics.’ These are built-in 5 characteristics that can be written and read from the client (in this case, Field Day) and the server (the LBB). These characteristics have set UUIDs, so that means I can’t use the custom UUIDs I set for the RBL’s sensor. After browsing some of the android SDK code from Punchthrough (the people that made the LBB), I was able to determine the UUIDs that are used for the characteristics. Since Field Day is going to have to determine what kind of device it is connected to, I redesigned it so that the fragment is cleaner and doesn’t do any of that work. There are separate classes for a Bluetooth Sensor, GATT Client, and Bluetooth Service. The Service talks to the Sensor and the Sensor talks to the GATT client and determines what type of device it is connected to by checking the UUIDs. All the Fragment does now is say when it wants to write a message or read a message.

Punchthrough has sample code available for arduino and reading and writing the scratch characteristics. The examples along with the SDK has proven helpful when rewriting the android code to accommodate for the Bean. Unfortunately, not many people have used the Bean with Android. All of the examples are with iOS. Field Day is able to read scratch characteristics just fine from the Bean, but is currently unable to write to any characteristics. One huge problem with the bean is that everything is Bluetooth. It’s not like arduino where you plug in the device to your computer to upload code and power it on. This means that the Bean can only be connected to one thing at a time and thus, is really hard to debug. Typically when debugging arduino devices, I’m able to plug the device into my computer and watch the Serial monitor for debugging code I’ve added in. With the Bean, I cannot. It’s blind debugging and it sucks. I’m still working on figuring out how to write a characteristic and have the bean read that characteristic, but I think I’m getting closer. I hope to finish that this week.

BLE in Arduino and Android
by on with No Comments

I have accomplished quite a bit within the past week, if I do say so myself. Last week, I said that I had begun working on the BLE part of Field Day. Well, after some struggle I finished it! I am able to send a request to the Arduino device and upon receiving that request, the Arduino device sends a message back to the android device and writes to he screen (woo!). But the code is not pretty. Android provides a Bluetooth Low Service example in Android Studio, but it uses deprecated code. I researched and was able to modernize it.

Bluetooth Low Energy is pretty complicated as a service. It uses something called GATT (Generic Attribute Profile). The bluetooth server (arduino device) has a GATT profile and within that GATT profile there are services and services have characteristics. Each service can have multiple characteristics, but there can only be one TX and one RX (read and write) characteristic per service. I learned this the hard way. You can see a diagram of what I’m talking about below.

For the arduino side, we use Red Bear Labs BLE shield. There is a standard BLE library for the arduino but it’s really complex. I’ve read over the code multiple times and I’m just now grasping a little bit of it. RBL has constructed a wrapper for that code. They broke it down into 10 or so different functions, which I must say is mighty useful. I used that when constructing the Arduino code.

During my coding of the BLE on android I discovered that most of our Fragments of sensors are all the same except for the names of them. After talking with Charlie we’ve decided that we’re going to get rid of the individual fragments for each sensors and just have one that is ‘Bluetooth Sensors.’ I’m working on moving the BLE code to that setup now. Hopefully I will be done with it within the day.

 

GATT

1 2