SolarNode plugin system

A long time ago SolarNode development switched from a traditional style Java application to a modular OSGi based one. One of the reasons for the switch was the premise of allowing users to easily download and use SolarNode modules without having to restart the SolarNode application. In OSGi installing, starting, stopping, and uninstalling modules (called bundles in OSGi) is fairly trivial. What SolarNode has lacked, however, was any friendly UI to allow users to browse the available modules and install the ones they want.

Finally that is about to change. The first step to enabling this feature was the deployment of a SolarNetwork-hosted software repository. There are a few open source packages out there, I ended up using Apache Archiva. Now SolarNetwork has a Maven-compatible online software repository to host its own modules, and the build infrastructure has been updated to easily publish modules as they are developed.

Browsing the SolarNetwork Repository

The next step was to make use of the OSGi Bundle Repository (OBR) developed as part of Apache Felix. OBR provides a way to resolve OSGi bundles along with their dependencies. Unfortunately, Archiva does not publish OBR metadata so I turned to another Apache project, Karaf Cave. Cave can act as a OBR proxy to a Maven repository, in which case it scans the Maven repository and generates OBR metadata files from which the OBR resource URLs point back to the artifacts in the Maven repository.

The final step was to develop some SolarNode bundles to enable configuring OBR URLs and a UI to allow users to browse the available software and install or remove individual modules. You can see the result of this work in the following demonstration:

This is a big step forward both in terms of ease of use for users and the SolarNode software development process!

Energy Datapalooza!

My name is John Gorman and I was one of the folks initially involved in starting the SolarNetwork platform, and am a director of Greenstage Power Ltd. SolarNetwork is a software platform for the the acquisition  storage, visualization and analysis of energy data from distributed and heterogeneous sources. In fact, it will do more than that now, thanks to Greenstage Power's efforts, but this is my first Greenstage Blog entry,  so hello!

Back in May 2012, I was invited to attend an Energy Data Jam at Stanford University, to help brainstorm how the use of open government data in the U.S. could serve as a catalyst to innovation in efficient energy use and conservation. Split into groups, we dreamed up and presented to the full group (about 40 people) a dozen or so "apps" that used government datasets.  At that meeting, U.S. CTO Todd Park told us it wasn't just about brainstorming, it was about really coding these apps, in 90 days.

Following up from the Energy Data Jam,  I was invited to represent SolarNetwork at the Energy Datapalooza in Washington DC on Oct 1, 2012. This was the conference organized by the OSTP (Office of Science and Technology Policy) an arm of the White House whose mission is to drive innovation and advise the president on technology changes and opportunities. The Energy Datapalooza was focused on celebrating the release of a number of open APIs allowing access to government data sets, and to highlight the entrepreneurs who were already starting to leverage those APIs for new products and services. From census data, to auto efficiency standards, to near real-time electricity usage, the APIs exposed a huge amount of valuable data and developers were busy doing mashups and apps that used it. A lot of credit should go to two guys who were really energetic in making this a reality: CTO Todd Park and EDI (Energy Data Initiative) Director Ian Kalin. Their passion for opening up data as a  catalyst for innovation and their bold, aggressive and open approach to welcoming experimentation is really appreciated. I think that once the GreenButtonConnectMyData API is opened up, (to be released soon) we're going to see some very interesting applications appear. One very interesting speaker at the Energy Datapalooza was Dr. Steven Chu, Secretary of Energy. A Nobel Prize winner in Physics, Dr. Chu spoke about the role of data streams and core technology as a fuel for innovation, and mentioned the impending falling prices of both PV and deep-cycle storage, with some very interesting applications coming up. We are of course eager to hear more, so we'll stay tuned!

SolarNode web configuration

Matt has been improving the usability of the SolarNode and SolarNetwork software and as a result, SolarNodes now support web based configuration!  Check it out:

Introductions, and expansion on minimalist Debian 6 SolarNode setup

This is my first post on the Greenstage blog, so a brief introduction is in order. I've been developing the SolarNetwork platform since meeting John Gorman in mid-2008. I was already interested in renewable energy, and a proponent of open source software, so SolarNetwork was a natural fit. John introduced me to Philip Court later that year, and eventually we joined forces formally as Greenstage Power, Ltd. with myself as CTO. SolarNetwork has already evolved considerably in many exciting ways over these past few years, and I look forward to seeing where the future takes it.

As for the original impetus for this post, I've expanded on Philip's previous post on setting up Debian 6 for a SolarNode on an eBox 3300 mini computer. Some of the improvements in this guide are:

• Stock i386 Debian kernel used
• OpenNTPD for time synchronization (instead of usual ntpd)
• Monit for process monitoring
• busybox-syslogd in-memory cyclic syslog
• SolarNode platform uses a ramdisk for logging, temporary files, and the embedded database

These last two points greatly reduce the number of writes the system performs on disk, with the idea of reducing writes to the flash-based media used as the system's main filesystem.

The guide is available on the SolarNetwork Wiki. The entire system described by that guide is also available as a downloadable disk image that you can easily copy to a SD card or USB stick.

Minimal Debian install on a Vortex86

For our SolarNode installations (part of the SolarNetwork) we use a minicomputer with a Vortex86.  This is a system on chip (SoC) x86 compatible CPU which has all the bells and whistles on one piece of silicon.

JrMX with Vortex86 inside

We had until recently been installing a standard Ubuntu distribution which was working very nicely.  Problem was this distribution was full of many unneeded software modules.

Over the break I finally got around to sorting out a bare bones installation that only contains what we need (no GUI required!).  Not only does this increase performance and reduce disk space requirements, but it enhances security by keeping deployed software to a minimum (less software equals less security updates).

Here is the process we have used to build a bare bones SD boot image with the latest native WiFi support (no ndiswrapper here) on a Norhtec JrMX.  Read on... 

First up: Find yourself an Ubuntu or Debian based PC to prepare your bootable SD card.  Make sure you have unetbootin or similar installed (usefull for creating a bootable SD or USB stick from an iso).

# sudo apt-get install unetbootin

Use unetbootin to create a bootable SD card (I used a 4GB) using this Debian squeeze mini iso.

Now boot the JrMX (with network cable connected) from the freshly made bootable SD card and complete the installation process (could take a while with lots of downloading).

Once finished, there will likely be some additional packages you will want to install e.g. ssh-server, ntp, wicd, wicd-curses etc.

# apt-get install ssh-server

# apt-get install ntp

# apt-get install wicd

# apt-get install wicd-curses

FYI: We use wicd-curses to configure WiFi from command line (instead of the NetworkManager GUI or similar).

Now, get Ready for the Kernel upgrade and WiFi driver install...

Squirt the stuff (Kernel and WiFi Driver) across to the JrMX from your desktop (connected using network cable at this point in time).
e.g.

$ scp linux-image-2.6.34.10-vortex86-sg_1.2_i386.zip  philipc@10.1.1.11:
$ scp RTL8188_8192SU_usb_linux_v2.6.6.0.20110401.zip philipc@10.1.1.11:

Now, on the JrMX box (ssh in or login using physical keyboard and monitor), install the new kernel and the kernel headers:

# unzip linux-image-2.6.34.10-vortex86-sg_1.2_i386.zip
# dpkg -i linux-image-2.6.34.10-vortex86-sg_1.2_i386.deb
# dpkg -i linux-headers-2.6.34.10-vortex86-sg_1.2_i386.deb

Now Install RTL8188SU Linux driver (for WiFi), but first we need to sort out the missing build link to the header source files:

# rmdir -–ignore-fail-on-non-empty /lib/modules/2.6.34.10-vortex86-sg/build

# ln -sf /usr/src/linux-headers-2.6.34.10-vortex86-sg /lib/modules/2.6.34.10-vortex86-sg/build

Download and unzip the latest RTL8188SU driver (which is actually the same as RTL8192SU driver).  The one I specifically used is here, or you can get it from the realtek site. 

#unzip RTL8188_8192SU_usb_linux_v2.6.6.0.20110401.zip

Note: This works great, but readme instructions from the older rtl8188_8192.zip (this one is from the robosavvy site) are much more informative.  Follow the readme instructions which are summarised below ('make' and 'make install' basically)...

Note: You will also need the following to build the drivers (if you want, remove with apt-get remove at the end):

# apt-get install build-essential

Extract, build and install WiFi drvier from unzipped source dir:

# cd rtl8712_8188_8191_8192SU_usb_linux_v2.6.6.0.20110401/driver
# tar -xzf rtl8712_8188_8191_8192SU_usb_linux_v2.6.6.0.20110401.tar.gz
# cd rtl8712_8188_8191_8192SU_usb_linux_v2.6.6.0.20110401
# make
  ...
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/solar/rtl8712_8188_8191_8192SU_usb_linux_v2.6.6.0.20110401/driver/rtl8712_8188_8191_8192SU_usb_linux_v2.6.6.0.20110401/8712u.mod.o
  LD [M]  /home/solar/rtl8712_8188_8191_8192SU_usb_linux_v2.6.6.0.20110401/driver/rtl8712_8188_8191_8192SU_usb_linux_v2.6.6.0.20110401/8712u.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.34.10-vortex86-sg'

# make install
install -p -m 644 8712u.ko  /lib/modules/2.6.34.10-vortex86-sg/kernel/drivers/net/wireless/
/sbin/depmod -a 2.6.34.10-vortex86-sg

# shutdown -r now

NOW LETS GET THAT WiFi GOING!

Check a few things:

# ifconfig wlan0 up
# iwlist wlan0 scan
# iwconfig

Configure the WiFi connection (use wicd-curses, it's like NetworkManager, but a command line tool! Just follow your nose...)

#wicd-curses

Yea Ha!!!! It all works :)

You may want to turn off a few other things as well, but this is a great starting point for a minimal install.

Enjoy!

Note: Please make sure you see Matt's update on this post as well.