Next Step Using Docker for Development

I'm trying to use Docker to build my local development environment on my Mac. (I don't want to have to rebuild my laptop as I try new things and experiment. That usually dirties up my laptop.) To that end, I'm working with two containers to implement different technology stacks for development.

My ideal set up would be to implement my entire developer experience in Docker. I would like to run Emacs out of one container, run tech stack tooling out of another container, and pull my Git repos into the tech stack repos as necessary. (Previously I've written about running GUI apps out of Docker containers on a Mac. Since that writing, I have upgraded to Docker for Mac from Docker Toolbox. Since that upgrade, I have been unable to get GUI apps to run out of a Docker container.) For my current purposes, I am running my tech stacks out of Docker containers, mounting my source code from local folders into the containers, and running Emacs locally.

The first container I created was for .NET Core development. This environment allows me to do .NET Core development natively on my Mac. I'm developing in .NET Core with Emacs. I'm also trying to use Omnisharp with Emacs. Since Omnisharp is a work in progress, I wanted to set up this development environment in a way that makes it easy to blow away and restart. I used the Microsoft supplied development image, and it works great for building .NET Core projects. I'm still struggling to get it to expose ports correctly, both for .NET apps and Omnisharp.

The second container I created is for NodeJS work. I used the NodeJS image to create my container. Right now I'm using this container for running AngularJS 1.x apps, and it is working pretty well. The ports are exposed correctly and I can access the apps externally from the container.

Since this is a work-in-progress, I'm keeping my "notes" in the form of Dockerfiles and shell scripts. These are being stored in this Github repo. This work may be of some use to others as they build similar environments. If nothing else, I should be able to find my previous experiments.

Product Management and Software Development

I recently attended a training session on writing requirements that was geared for product managers. I sat in on the third day of a 3-day course. The day consisted mostly of how to create and prioritize requirements for feeding into development teams.

The instructor focused on the following aspects:

• Using market research to drive priority.
• Requirements should focus on a problem and not implementation.
• Requirements need to include personas, a problem, and a frequency that the problem occurs.

All of this was good information and made sense to me.

There was an undercurrent in many of the statements that seemed to reinforce an us-versus-them mentality between product managers and software developers. I think this undercurrent missed the mark and is probably doing more harm than good.

I have seen a lot of success over the last decade as a software developer at Clearent expressly because there has been a strong relationship between product management and software development. As the manager of a software team, I have pushed my team to learn and understand our industry along with our particular approach to doing business. I joke with my boss that members of my team spend more time learning the business than they do any of our technology. This learning has allowed my team to work as a partner with product management to creatively build tools and solutions that set us apart in the industry.

On the other side, product management has taken a LOT of time to provide context and education for my team so that they can get smarter about our business. All of our major development efforts start with several discussions about the business needs and the landscape of the problem we are trying to solve. My team is never in the dark about the motivations behind the changes we are making.

Together, we can have productive conversations about technical debt and business needs. The development team can raise concerns over technical debt. The product management team can talk to business needs that will help propel us forward. Because the dialog is good, we always find a good balance between the two concerns.

Healthy dialog between product management and development is the key to successful software development projects. This dialog exists in the close relationship we have at Clearent between product management and software development.

I hope it exists in your job too!

Building a Cordova App

In an earlier post I wrote about the golf score tracking application I created to learn ASP.NET Core and AngularJS. Since the initial inspiration was my son playing golf, I thought it would be good to put the application on his tablet so he could track his own scores. Cordova looked like the technology to allow me to turn my "experiment" into something my son could practically use.

I'm going to focus on my experience with Cordova and getting a mobile app running. (I'm writing this post mostly so that I don't forget the little things I ran into while getting things to work.)

Cordova is an Apache open-source project for building mobile applications using web-based technologies. I have heard about this technology for a few years at several tech conferences that I attend. The appeal of Cordova is that it should allow a developer to publish an application to multiple mobile platforms using the web-based technology that is already known. Cordova was born out of a technology called PhoneGap.

The Cordova utilities run on Node.js and can be installed using the Node Package Manager (NPM). The Getting Started page of the Cordova site has all of the basic steps necessary to get Cordova installed and the first application created. Here are the steps I followed:

1. Install Cordova.

   npm install -g cordova

2. Create a Cordova application.

   cordova create com.darrenhale.golfscoretracker golfscoretracker

   This step is different than the step listed on the Cordova Getting Started page. I was required to include the "com.darrenhale.golfscoretracker" portion to generate the application structure.
3. A Cordova application has to have "platforms" added for the deployment targets. I wanted my application to function in the browser, on Android, and on iOS

   cordova add platform browser android ios

4. Migrate the "web" application into the Cordova application structure. For me this meant:
   1. Copying my index.html page into the www folder of the Cordova application.
   2. Copying my Javascript files into www/js folder.
   3. Copying my bower_components folder into the www folder.
   4. Copying my css folder into the www folder.
   5. Copying my styles folder into the www folder.
   6. Making any path changes to the files referenced in index.html
5. At this point, I tried running the application in the browser.

   cordova run browser

Up to this point, things were great! The application worked when I called:

cordova run browser

I had also started my web API application to have a back-end process. Everything worked as expected! (I expected it to work since the AngularJS app worked in a browser already.) My next step was to run it for a mobile platform.

In my excitement, I quickly executed:

cordova run android

Guess what happened? Nothing. It was late, and I realized I was not thinking straight. It dawned on me that I needed an Android emulator to make this command work. Let the adventure begin!

At my next opportunity to work on the project, I immediately went to get the Android SDK and its corresponding emulator. I didn't realize how much the state of mobile development has changed, and I was surprised to find that the first thing I found was Android Studio. I really didn't want to download a full development environment. After some poking around I was able to find just the Android command line tools. I downloaded the package for Mac OS X and unzipped the package into a folder on my system.

I created a test device to emulate for running my application. Through trial and error, I came up with the following configuration:

• Targeted Android 6.0 - API Level 23
• CPU/ABI of ARM (armeabi-v7a)
• RAM of 512
• VM Heap of 32
• Internal Storage of 200 MiB
• Emulation Options: I selected Use Host GPU

I could now successfully start the Android emulator and interact with it. (It is important to note, the emulator is VERY SLOW to start completely. Give it time to start up.)

With the emulator running, I executed:

cordova run android

The Cordova utilities packaged my application, pushed it to the emulator, and started the application. It worked! My application started up and functioned! I realized I needed to adjust the CSS in the app to work on a smaller display, but those changes were straightforward to make. After a little more trial and error, I had an application that functioned on a mobile device.

The last change I needed to make was to replace the API calls in the application with calls to localstorage. I opted to use localstorage for two reasons: first, I wanted my son to be able to use this application without any network connectivity, and second, localstorage is a mechanism that a Cordova app can use across multiple platforms. Localstorage is essentially a key/value store using JSON as the storage mechanism. Most of the data in the golf score app is pretty simple, so I figured I could make this work. The localstorage object has a setItem() method for saving data and a getItem() method for retrieving data. I was able to replace my API calls with localstorage calls fairly easily, but the app didn't work right away. The trick I was missing was that I needed to convert my objects into a JSON format to save them and to parse JSON into my object when retrieving them. I accomplished this with JSON.stringify() and JSON.parse() respectively.

My path for learning Cordova is not unique, and I'm sure many of the challenges I have experienced aren't challenges for experienced mobile developers or Javascript developers. It was still a great learning experience and it has me excited to do more Javascript coding!

Innovation and Competition in St. Louis

I read Walter L. Metcalfe Jr.'s editorial this morning in the St. Louis Post-Dispatch. In his editorial, he calls on St. Louis to start a new competition similar to the one that brought us the Gateway Arch to spur collaboration and innovation. While reading this article, I couldn't help but think of hackathons.

Hackathons are generally time-limited competitions in which technology specialists work to create prototype solutions to problems. Sometimes these problems are business-oriented, but quite often the problems are civic focused. These competitions require teams of people to work together to solve the problem.

My company Clearent and the University of Missouri - St. Louis are partnering for a competition in the same spirit as Mr. Metcalfe is calling for, but on a smaller scale. We are sponsoring a hackathon called HackEdu. This is an inaugural event in which we are going to bring together educators, technology specialists, and business people to solve some of the most pressing challenges in education using cutting edge technologies. HackEdu has several goals in mind.

First and foremost, we want to introduce many different constituent groups to each other, and help them learn how to collaborate together effectively. Quite often we see divisions between educators, business people, and technologists. This division limits innovation, especially in the education sector. Within the context of the HackEdu, we want to help these different groups learn how to work together effectively.

Another goal of this event is to teach participants how to to function in a hackathon. Many participants in a hackathon don't have the tools necessary to implement a process that helps them effectively achieve their goals. During the HackEdu event, we plan to introduce participants to a methodology that Clearent uses to develop software. Our process focuses on business value, features, and functions. This approach allows us to break down business goals into smaller features and ultimately individual functions that allow us to incrementally release software that provides value quickly.

The obvious goal of the HackEdu event is to solve a problem related to education. A specific problem will be announced at the event, but it will focus on an educational need that could benefit from a technology-oriented solution.

Because much of the HackEdu event is directed toward teaching people how to collaborate and create functional solutions, we are offering the winners of the HackEdu tickets to Globalhack VI. Globalhack is an annual hackathon where numerous teams come together to solve a civic-oriented problem. Our hope is to prepare people to effectively compete in the Globalhack event this year.

I believe that the HackEdu competition and the Globalhack event are preparing people in St. Louis to participate in a competition like the one Mr. Metcalfe describes in his editorial. If nothing else, I think we are building the skills necessary to continue to build the St. Louis area as a colloborative, innovative community!

ASP.NET Core App with AngularJS

I've been trying to learn the new ASP.NET Core technology along with AngularJS. I always find it easier to learn new technology if I have some problem that I'm trying to solve. The problem that presented itself to me was a golf score tracking application. My middle son plays golf for his school team. I thought it would be great to build a mobile app to keep track of his golf score on the course. Since I've never been great at UI work, I thought this was a good opportunity to learn a front-end technology too.

I created an MVC 6 web API that exposes REST endpoints for working with a golf score. My intention with the application is to mimic a golf score card. The API went together pretty quickly. I kept the API simple: I have a single controller that exposes GET, PUT, POST, and DELETE methods on the Score object. The Score object holds some basic information about a round of golf (i.e. the course name and a description) along with a collection of scores for holes. The controller is saving data to a text file. I used a text file instead of a database to simplify the learning process. (Another motivation for saving to a text file instead of a database was because DNX Beta 4 did not have the ability to work with a SQL Server database, and I didn't want to hunt down a compatible technology.) Over the last several months, I've upgraded the back-end portion of the app to stay up-to-date with the ASP.NET Core changes, bringing it up to RC1.

The front-end of the application is an AngularJS application using Bootstrap for styling. At the time I created the initial application, I also wanted to learn some of the modern web tooling, including Bower. I used Bower to lay down the initial AngularJS application, but I quickly abandoned keeping up with Bower becase I had my hands full just learning AngularJS and Bootstrap.

I experienced two main difficulties creating the web application. First, I had a difficult time understanding AngularJS routes, views, controllers, and the relationships between them. Once I started to "get" how things were wired together, I was able to I went through several passes to get an application functional. The Javascript code is not pretty and probably not well organized. My focus was on understanding learning AngularJS and not making things "pretty."

I'm not a developer that is good at creating large amounts of documentation for my code. I generally want code to stand for itself. To that end, this project is in Github here. Hopefully the code does actually speak for itself.

In future passes, I would like to clean up the Javascript and add unit testing to both the API and Angular pieces. I also plan on upgrading this app to RC2 of ASP.NET Core when it is released. (I may try to do this sooner if time allows.)

Running GUI Apps in Docker on Mac OS X

I've been trying to figure out how to run GUI apps in a Docker container on my Mac running OS X El Capitan. As a developer, I try out many different software packages and configurations. I would love to be able to try out certain things in a well-contained environment where I won't leave bits and pieces lying around if something doesn't work or if I get bored with the package. I have also been enamored with using Docker for development environments to make them portable and self-contained. After many searches and experiments, I finally stumbled on what I needed to do.

I am using the Docker Toolbox to run Docker on my Mac. I have also used Boot2Docker, but the Docker Toolbox experience is a little smoother. Either option will work to get Docker containers running on your system.

One of the required pieces I had to install was XQuartz. This is an X11 windowing system that runs on the Mac. I was originally worried about installing this package, concerned that it would interfere with the X11 windowing system built into OS X. So far, I haven't had any trouble with it.

To try out a GUI app, I wanted a basic Linux container with an obvious GUI app that is not installed on my Mac. I created a basic Dockerfile defining an image based on the latest distribution of Ubuntu. I then install Firefox into the image. The image starts up at a bash prompt, which allows me to run programs. Here is that docker file:

FROM ubuntu:latest

RUN apt-get update && apt-get install -y firefox

CMD /bin/bash

Getting Docker to create an image and run a container based on that image was pretty straightforward with the Docker Toolbox for Mac OS X. The hardest part was trying to get the running container to bind the DISPLAY environment variable to the XQuartz server. This link finally broke things open for me. A comment from slobo on January 22, 2015 has the secret sauce. In an xterm window started under XQuartz, run the command

socat TCP-LISTEN:6000,reuseaddr,fork UNIX-CLIENT:\"$DISPLAY\"

Then, when starting the container, set the DISPLAY environment variable to the IP of the Docker virtual machine. (This IP address is usually a 192.168.x.x value.) I found this IP address on my Mac by typing

ifconfig

and locating the vboxnetn entry (where n is 0 - 9). For me, it looked like:

Using the IP address 192.168.99.1, I ran the following command to start a running container:

docker run -it -e DISPLAY=192.168.99.1:0 firefox

(firefox is the image name for the Ubuntu image running Firefox I defined with the Dockerfile above.) When I got to the bash prompt inside the running container, I ran Firefox, and the UI appeared like another window in OS X!

I am just starting my journey with Docker, but I'm excited about the possibilities it offers.

UPDATE:

Benny Cornelissen (@whieee) has a blog post that does a really good job integrating XQuartz and the instantiation of containers into the bash shell.

Setting Future File Permissions in Linux

Here is a link to a StackOverflow article outlining how to do this.

I actually found something that so far does what I asked for, sharing here so anyone who runs into this issue can try out this solution:

sudo setfacl -Rdm g:groupnamehere:rwx /base/path/members/
sudo setfacl -Rm g:groupnamehere:rwx /base/path/members/

R is recursive, which means everything under that directory will have the rule applied to it. d is default, which means for all future items created under that directory, have these rules apply by default. m is needed to add/modify rules.

The first command, is for new items (hence the d), the second command, is for old/existing items under the folder. Hope this helps someone out as this stuff is a bit complicated and not very intuitive.