Long Day’s Journey Unto Ice
by on with No Comments

Today began at 7:30 when Charlie and Emi left to pick up the rental van at the airport in Egilsstaðir, roughly 30-45 minutes away. Sanne, who runs and maintains the house at Skalanes, drove them – and as much luggage as we could pack in – out to the parking area where we had one rental car waiting. They transferred people and luggage into the rental and drove off. The rest of us ate breakfast, cleaned up the house as best we could to help out Sanne and the others who run Skalanes (a group of 30 is arriving at Skalanes tomorrow!), and began to load up Sanne’s vehicle with all of the luggage we had not been able to send with Charlie and Emi. Once the luggage was loaded, we began walking out the drive while Sanne drove the luggage to the parking area and then turned around to pick us up and drive us the rest of the way. At the parking area, we were faced with a cold and windy wait as Charlie and Emi drove back. We amused ourselves with word games and hypothermia dances until finally they arrived.

Loading up the vehicles with luggage and humans took only a few minutes, and then we were off on our long drive back to Vik. We stopped for a few minutes at Glacier Lagoon so Marketing could get some nice picturesque footage of us in Iceland and briefly saw a seal – they occasionally swim up into the lagoon, which is apparently close to 200 meters deep due to the icebergs. Other than that, the drive was long and pretty uneventful, with stops only for restrooms and other necessities. At one of the gas stations we learned that today is a holiday here in Iceland (Whit Sunday, the day before Pentecost), and we were concerned that the grocery store here in Vik would be closed, preventing us from getting groceries for tomorrow. We made a short detour to Klaustur, where we knew there was a gas station/convenience store that would have some basic groceries available, and picked up materials to make some quick Mac and Cheese for dinner tonight and sandwich materials for lunch tomorrow.

When we pulled into Vik we discovered that the grocery store is in fact open today and we needn’t have worried, but the early shopping trip did allow us to begin working on dinner much sooner after we arrived. Everyone was tired and hungry, since we pulled into the hostel at 7 pm and had left Skalanes at 8:30 (or earlier) this morning. It’s after dinner now, and Charlie and Neil are working on getting the elevation platforms and drone ready for tomorrow, when we’ll be returning to the Sólheimajökull glacier. Rather than collecting soil samples like last time, we’ll be climbing on the glacier itself to collect geographical and volumetric data we’ll use to chart its recession over time. Nic is down with a nasty cluster headache, which is preventing him from helping get the drone ready. Yesterday I finished modifying and debugging the elevation platforms’ code, but now Charlie is trying to debug an issue with Field Day, the Android app we use to gather data from them. All in all, it’s a pretty regular night for us here, with a group playing Euchre in one area and tech gear exploded across another space.

Tomorrow we leave the Puffin Hostel at 9 am to meet our glacier guides at 10 and begin collecting data on the ice.

Sending data back and forth
by on with No Comments

As you remember, last week I finished the new data model implementation within Field Day and downloading remote database data. Field Day writes sensor database to the local database respecting the new model — populating fieldday_streaming, fieldday_reading, and fieldday_spot.

The big hurdle that still needed to be hopped over was sending those newly populated tables back to the remote database. This week I got over that big hurdle. The user can now upload the local tables to the remote tables. Field Day gets all of the rows in the tables I mentioned above, starting with fieldday_spot because the other two have foreign keys that reference the spot table. Moving the rows from the reading and streaming table is simple, but moving the spot table has different characteristics. If people are pushing to the database at the same time, we have to make sure that we are only inserting one of each ‘trip, site, sector, and spot’ which make up the primary key in the database. Field Day takes the ‘tripID, siteID, sectorID, and spotID’ of each row in the local database and queries the remote database asking for the count where a line has those four columns that equal the values in the local table. If the count comes back zero, then Field Day inserts the row. Each tables updating is wrapped up in a transaction, so if something goes wrong Field Day rolls back the transaction and sets the database to it’s previous state. Once the inserting has successfully finished, the database in the local table is copied to external storage in an /archive directory and the tables are dropped and created again so they are empty.

I moved ‘Take a Sample’ on the main screen to ‘Sampling’ and created a new option under that menu for ‘Database Actions.’ The downloading from remote database activity was previously under the Settings option, but since the application is basically doing the same thing in each, I merged them in to one activity. There are three buttons for the user to choose from (which you can see in the picture below) — setup local database, upload tables, clear local database. Upon the press of any of the buttons, the user is asked to confirm.  When the user selects clear local database, just like after the user upload the tables, the local database is copies and wiped.

A couple other small things that I’ve worked on this week are integrating the Google Drive Android API and fixing small things with the sensor sampling activity. As for the Google Drive API, I’ve gotten Field Day to the point where it asks for permission to access Google Drive, but doesn’t do anything after that. To my knowledge, this API only allows the app to interact with files that were created by the application itself. I’m not 100% sure of that so I’m still working on it. Small things that are fixed are the user is notified if they haven’t set up their local database with a remote database, which means they can’t write anything to the local database. When a Spot number is put in the spot edit text on the sampling activity, Field Day checks to make sure that spotID doesn’t already exist in the local database if it does, then it doesn’t write to the database.

 

device-2016-05-19-094627

Light Blue Bean!
by on with No Comments

In my last post, I said that I was going to switch all of our individual fragments of sensors to one that is ‘Bluetooth Sensors.’ Since then, I have finished that and redesigned even more of Field Day.

After I finished switching all the fragments to one, I dove into working on BLE connection with the Light Blue Bean (LBB). Unfortunately, the LBB does not do Bluetooth the same way the Red Bear Labs shield does. The LBB uses something called ‘Scratch Characteristics.’ These are built-in 5 characteristics that can be written and read from the client (in this case, Field Day) and the server (the LBB). These characteristics have set UUIDs, so that means I can’t use the custom UUIDs I set for the RBL’s sensor. After browsing some of the android SDK code from Punchthrough (the people that made the LBB), I was able to determine the UUIDs that are used for the characteristics. Since Field Day is going to have to determine what kind of device it is connected to, I redesigned it so that the fragment is cleaner and doesn’t do any of that work. There are separate classes for a Bluetooth Sensor, GATT Client, and Bluetooth Service. The Service talks to the Sensor and the Sensor talks to the GATT client and determines what type of device it is connected to by checking the UUIDs. All the Fragment does now is say when it wants to write a message or read a message.

Punchthrough has sample code available for arduino and reading and writing the scratch characteristics. The examples along with the SDK has proven helpful when rewriting the android code to accommodate for the Bean. Unfortunately, not many people have used the Bean with Android. All of the examples are with iOS. Field Day is able to read scratch characteristics just fine from the Bean, but is currently unable to write to any characteristics. One huge problem with the bean is that everything is Bluetooth. It’s not like arduino where you plug in the device to your computer to upload code and power it on. This means that the Bean can only be connected to one thing at a time and thus, is really hard to debug. Typically when debugging arduino devices, I’m able to plug the device into my computer and watch the Serial monitor for debugging code I’ve added in. With the Bean, I cannot. It’s blind debugging and it sucks. I’m still working on figuring out how to write a characteristic and have the bean read that characteristic, but I think I’m getting closer. I hope to finish that this week.

BLE in Arduino and Android
by on with No Comments

I have accomplished quite a bit within the past week, if I do say so myself. Last week, I said that I had begun working on the BLE part of Field Day. Well, after some struggle I finished it! I am able to send a request to the Arduino device and upon receiving that request, the Arduino device sends a message back to the android device and writes to he screen (woo!). But the code is not pretty. Android provides a Bluetooth Low Service example in Android Studio, but it uses deprecated code. I researched and was able to modernize it.

Bluetooth Low Energy is pretty complicated as a service. It uses something called GATT (Generic Attribute Profile). The bluetooth server (arduino device) has a GATT profile and within that GATT profile there are services and services have characteristics. Each service can have multiple characteristics, but there can only be one TX and one RX (read and write) characteristic per service. I learned this the hard way. You can see a diagram of what I’m talking about below.

For the arduino side, we use Red Bear Labs BLE shield. There is a standard BLE library for the arduino but it’s really complex. I’ve read over the code multiple times and I’m just now grasping a little bit of it. RBL has constructed a wrapper for that code. They broke it down into 10 or so different functions, which I must say is mighty useful. I used that when constructing the Arduino code.

During my coding of the BLE on android I discovered that most of our Fragments of sensors are all the same except for the names of them. After talking with Charlie we’ve decided that we’re going to get rid of the individual fragments for each sensors and just have one that is ‘Bluetooth Sensors.’ I’m working on moving the BLE code to that setup now. Hopefully I will be done with it within the day.

 

GATT

Handlers handling handles
by on with 1 Comment

After some testing of the Handler implementation in Field Day, I’ve determined that it is actually working! I was finally able to look at the data that I had been writing to the database.

Databases in Android are stored in internal (private) memory. There are Android apps that let you look at the files on your device, but only for external storage. The internal storage is private. What I ended up doing was adding a function to write the database from internal storage to external storage. Once it was on external storage, I used a SQLite Viewer Android app to look at the database and it looks good!

I did some research into the application state when the device screen is turned off. In our old application, Seshat, the activities would die when the screen was off. That’s not good. We want to be able to turn off the screen and hang the Android device from our person some how. It’s useful to have our hands free. After a couple hours of research, I determined the libraries we want to look at are in the PowerManager library on Android, particularly the PARTIAL_WAKE_LOCK state. I wrap the methods I want to happen when the screen is off in what are essentially ‘start’ and ‘stop’ methods. The methods were not executing with the screen off, so I researched some more. I think what will ultimately happen is switching the SensorSampleActivity to a Service. A Service in Android is one of the top priority processes. It’s one of the last to die to if the device needs more memory. It can continue running even if the Activity the started it has died. Since the Activity is now working, I’m going to hold off in converting it to a Service.

The last thing I worked on this week was Bluetooth LE in Android. Tara gave me a BLE shield with an Arduino attached to test my code with those devices. Android provides a BLE example in Android Studio so I’ve been working with that. However, the example they provide has deprecated code so I’m working on modifying it to the new setup. It’s not working yet, but I’ve only been working on it for a couple of hours!

sqlite-fieldday
Sample of sqlite database on Field Day

Databases, Threads, and aesthetics!
by on with No Comments

After getting most of the logistics sorted out, I’ve finally been able to get back to working on Field Day. Not a whole lot has been done because I spent most of my time working on logistics but I was able to get a few things done or started. The first thing I did was get rid of the black background for the Sensor Sample activity. We don’t have to keep it this color I was just tired of the black. I made it green because it reminded me of grass and being in the field. The next thing I added was EditTexts for Site, Sector, and Spot like Seshat had. In the data model, we also added a ‘Trip’ column but I didn’t think that there needed to be an option for that on the main screen. I put in option in the Settings page since it’s going to be constant throughout the entire trip. In Seshat, we had an option to select how often the stream to the database. I added that in Field Day as a Radio Group with customized buttons from 1 – 60 seconds. I haven’t done anything with those values yet but the visual and backbone is in place. (You can see these edits in the two pictures below — Sensor Sample on the left and Settings on the right). Ignore the ‘Table name for readings database’ on the Settings page for the time being because I have figured that out yet.

fielddaysettings

Not much to report…
by on with No Comments

I haven’t done much in the last week, to be honest. It was kind of a crazy week for me so I had to focus on some Admin things. I’m still looking at the best method for threads in Android and thinking about the Notepad/Lab Notebook/Checklist aspect of Field Day and the best methods and classes to implement it. If there’s anything that could improve the Notepad from Seshat. That uses a Navigation Drawer but to my knowledge, that has been deprecated in the new APIs, so more research is to be done.

Best method for threads in Android
by on with No Comments

The past week I worked on making the classes that pertain to sensor sampling platforms more generic. Previously, I had been working on just getting them working using the Built-In sensor sampling fragment. The Built-In fragment uses the built in android sensors on the hardware device which uses the Android Sensor class. The aSensor class and other similar classes that we are going to use for all sensor platforms (Database reading and writing, sensor sample fragment frame, and the list adapter) were using Android Sensor class methods. I worked on getting rid of those methods and replacing them with more generic methods that can be used for all sensor sampling types.

I have also been researching the best method for creating a thread in android classes. The SQLite work I’ve been doing is almost finished except it’s writing to the database too quickly which is making my primary keys fail. I am looking for best practices for writing to a database in a thread. In Seshat (our old application), we had a feature where the user could select how often they wanted to write to the database (in seconds). I plan to move this feature to Field Day but I want to research the best approach. In Seshat, we used a runOnUiThread method. This may be the best approach but there are other options to consider — Service, AsyncTask, Handler, Runnable, etc.

Gitlab fixed, back on track
by on with No Comments

After my last post, I decided to dive into our Gitlab setup and see if I could fix the problem. In case you’ve forgotten, the problem was the latest pushed code was broken. So, our master branch was pointing to code that was not working when pulled and deployed. The pushed code had changes to the gradle and build setup, not java code, so that’s why it was pretty broken. None of us know git all that well (but let’s be honest, who does?) so it has taken us a little while to remedy our setup.

I dove in and fixed it learning a lot about git in the process. I basically had to create a bunch of temporary branches with my current working code and rewrite history — basically making it look like the last changes had never happened. I set the origin/master branch to my working code with changes.

After this debacle, we decided that we should be using branches more often, keeping the origin/master branch for production, working, thoroughly tested code.

Back to Android and SQLite
by on with No Comments

Since the beginning of the semester, I’ve been working on Field Day again and somewhat sorting out git. At the end of last semester, we were having trouble with git and we’re still trying to figure that out. We’ve decided that we’re all going to stick on the same Android Studio version and upgrade at times we decide as a group. Nic has the most updated version 1.5, so we’re all going to stick to that.

The latest code push was code that broke my Field Day setup, so all of the code I’ve been working on hasn’t been pushed yet. I’ve started a new branch of our git repo called SQLite work that I’ve been committing my changes to. As a result, I’ve been learning a lot of git and branching and I think this may be the way to do so we don’t end up with broken code in our ‘master’ branch.

The part of Field Day I’ve been working on is the SQLite database. Previously, we wrote all of our readings to a CSV file but that gets quite messy. So, in this new application we’ve decided to go to SQLite. I’ve written a class called ReadingsDatabase which is a SQLiteOpenHelper class. This is used to create, delete, upgrade, update, query, etc the database. I’ve also written an interface that will be used to communicate between the Sensor Fragment classes and the main activity. I’ve been using the ‘Built-In Sensors’ fragment to test because we haven’t created any of the other sensors to test yet. I believe it is working, but the problem I’m running into is the class is writing to the database too quickly which is causing the primary keys I’ve set up to fail. I’m looking into a way to slow down the writing to the database. Once we figure out the git problem, I’m going to push my code.

1 2