Interprocess Communication With Memory-Mapped Files, Part 1

Memory-mapped files are great concept that have been buried in various versions of Windows for years.  Finally they've been exposed and incorporated into .NET 4.0 and are available in Visual Studio 2010.

A memory-mapped file is essentially just a file, given a unique name-- no extension necessary-- that exists only in memory.  It can be used to cache a large file into memory for faster access and manipulation.  In addition, since they're accessible from the system level, these files can be used to allow multiple processes to communicate (called "interprocess communication").  It's this method of communication that I'd like to try to implement here.

Suppose we have two separate processes, one to perform work, and the other that's simply responsible for starting the first-- like a Windows Service.  In this situation, we're assuming that the Windows Service can't simply perform the work in a child thread, but relies on that other process to do it.

I've created two C# projects, with one intended perform "processing" and the other to launch and communicate with the first:

  • ConsoleAppListener
    Responsible for performing work and shutting down gracefully when it receives a "shutdown" message, after it has finished processing a full unit of work.  Multiple instances can exist, but each has a different "unique name" and a separate ConsoleAppController responsible for starting and monitoring each instance.
  • ConsoleAppController
    Responsible for starting ConsoleAppListener, ensuring it stays running, and sending a "shutdown" message instructing the ConsoleAppListener to shutdown gracefully when it's finished processing.  Multiple instances can exist, but each is responsible for monitoring a ConsoleAppListener with a different "unique name".
Since a memory-mapped file has a unique name and persists only while a process is using it, it is a strong candidate for meeting these requirements.  The first process to start, expected to be the ConsoleAppController, is responsible for creating the memory-mapped file.  The second process, the ConsoleAppListener, will look for a memory-mapped file with the same unique name it was provided in its command-line arguments.
Here's a rough order of execution, although many of these things can happen in parallel:
  1. ConsoleAppController starts, checks if a memory-mapped file exists with the name "SampleApp"
  2. ConsoleAppController reads the memory-mapped file, checks if a process ID has already been recorded. If so, this means ConsoleAppListener is already running and will now be monitored.  Otherwise launches a new ConsoleAppListener process with the "SampleApp" passed as a command-line argument
  3. ConsoleAppListener starts, creates or opens memory-mapped file with the name "SampleApp"
  4. ConsoleAppListener saves its process ID to the memory-mapped file
  5. ConsoleAppListener performs processing, checking periodically for the shutdown message, "-1"
  6. ConsoleAppController monitors, restarts ConsoleAppListener if it shuts down on its own (e.g. user closes the window)
  7. ConsoleAppController saves a shutdown message, "-1", to the memory-mapped file, instructing ConsoleAppListener to shutdown at an appropriate time (when it's not performing processing)
So in this scenario, the mem0ry-mapped file only needs to be large enough to store an int32, representing either the ConsoleAppListener's process ID or a shutdown message of "-1".  The direction of either message is implied, as the listener only cares about receiving a "-1" and the controller is only interested in a process ID (which is greater than -1).  For that reason, I consider this mechanism more of a passive, broadcast-based communication system, rather than an active, full-duplex sort of client-server arrangement.
Here's some of the code used by the ConsoleAppListener, responsible for listening for a shutdown message:
private static bool ShouldClose()
        {
            using (var stream = _memoryMappedFile.CreateViewStream())
            {
                var reader = new BinaryReader(stream);
                var message = reader.ReadInt32();
                if (message == CloseMessage)
                {
                    Console.WriteLine("Received shutdown message");
                    return true;
                }
            }

            return false;
        }
Here's part of ConsoleAppController, responsible for locating the memory-mapped file ("destinationAppName" is the name of the memory-mapped file) and sending a shutdown message to the ConsoleAppListener if prompted by the user:
        private static void WantProcessToShutDown(string destinationAppName)
        {
            if (!_isClientProcessRunning)
            {
                return;
            }

            var memoryMappedFile = MemoryMappedFile.OpenExisting(destinationAppName);

            using (var accessor = memoryMappedFile.CreateViewAccessor(0, MessageByteSize))
            {
                accessor.Write(0, CloseMessage);
            }

            Console.WriteLine("Sent message to shut down");
        }
I hope this preview gives at least some insight into the use of memory-mapped files for primitive interprocess communication.  In tomorrow's post I'll be providing all of the sample code.

Setting the Netduino's DateTime Automatically

Whenever the Netduino boots up or is restarted, you'll find that the date and time reverts to the default values.  Since the release of the Netduino Plus, ethernet access has been built in and provides a welcome solution to this issue.

The Network Time Protocol (NTP) provides an easy way to sync a device (such as your Windows computer) with a network time server.  By invoking an NTP server at start-up, the Netduino will automatically set its DateTime.

The following code was provided on a blog by Michael Schwarz and slightly modified to accept an additional (time zone) parameter:

using System;
using System.Net;
using System.Net.Sockets;

public static class Ntp
{
    public static bool UpdateTimeFromNtpServer(string server, int timeZoneOffset)
    {
        try
        {
            var currentTime = GetNtpTime(server, timeZoneOffset);
            Microsoft.SPOT.Hardware.Utility.SetLocalTime(currentTime);

            return true;
        }
        catch
        {
            return false;
        }
    }

    /// <summary>
    /// Get DateTime from NTP Server
    /// Based on:
    /// http://weblogs.asp.net/mschwarz/archive/2008/03/09/wrong-datetime-on-net-micro-framework-devices.aspx
    /// </summary>
    /// <param name="timeServer">Time Server (NTP) address</param>
    /// <param name="timeZoneOffset">Difference in hours from UTC</param>
    /// <returns>Local NTP Time</returns>
    private static DateTime GetNtpTime(String timeServer, int timeZoneOffset)
    {
        // Find endpoint for TimeServer
        var ep = new IPEndPoint(Dns.GetHostEntry(timeServer).AddressList[0], 123);

        // Make send/receive buffer
        var ntpData = new byte[48];

        // Connect to TimeServer
        using (var s = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp))
        {
            // Set 10s send/receive timeout and connect
            s.SendTimeout = s.ReceiveTimeout = 10000; // 10,000 ms
            s.Connect(ep);

            // Set protocol version
            ntpData[0] = 0x1B;

            // Send Request
            s.Send(ntpData);

            // Receive Time
            s.Receive(ntpData);

            // Close the socket
            s.Close();
        }

        const byte offsetTransmitTime = 40;

        ulong intpart = 0;
        ulong fractpart = 0;

        for (var i = 0; i <= 3; i++)
            intpart = (intpart << 8) | ntpData[offsetTransmitTime + i];

        for (var i = 4; i <= 7; i++)
            fractpart = (fractpart << 8) | ntpData[offsetTransmitTime + i];

        ulong milliseconds = (intpart * 1000 + (fractpart * 1000) / 0x100000000L);

        var timeSpan = TimeSpan.FromTicks((long)milliseconds * TimeSpan.TicksPerMillisecond);
        var dateTime = new DateTime(1900, 1, 1);
        dateTime += timeSpan;

        var offsetAmount = new TimeSpan(timeZoneOffset, 0, 0);
        var networkDateTime = (dateTime + offsetAmount);

        return networkDateTime;
    }
}

Here's a short list of some NTP servers:

  • pool.ntp.org (actually it's a collection of servers behind a single address)
  • time.nist.gov
  • time-a.nist.gov
  • time-b.nist.gov
Pass one of these server names as a string and call during your project's Main() method:
private static bool SetTime()
{
    var result = Ntp.UpdateTimeFromNtpServer("time.nist.gov", -4);  // Eastern Daylight Time
    Debug.Print(result ? "Time successfully updated" : "Time not updated");

    return result;
}
It's unfortunate that the Netduino doesn't support Microsoft.SPOT.ExtendedTimeZone.SetTimeZone() to allow a time zone to be specified (thus eliminating the need for that extra "timeZoneOffset" parameter I added to two of the methods) or even the Microsoft.SPOT.Time.TimeService.UpdateNow() method, which would make all of the code above obsolete. However, it's understandable some libraries needed to be stripped due to memory size limitations. I'm still a huge fan of the Netduino; it still offers a lot of functionality on a very small and cheap device.

Additional References:

Blast From the Past: GW-BASIC

A few weeks ago, I decided to clean up my bookshelves and finally toss a bunch of antiquated software development books.  Technologies like HTML 3.2, Visual Basic 6, and Visual J++ didn't survive this latest purge.  But there's one book that I just can't seem to toss, no matter how much relevance it's lost.  It's the GW-BASIC User's Guide and Technical Reference, which was included with my very first computer, a CompuAdd 8086.

GW-BASIC was a port of the BASIC (Beginner's All-Purpose Symbolic Instruction Code) language, and was one of the many BASIC products borne from the assembly line of early Microsoft.  It was packaged with earlier versions of MS-DOS, the operating system of choice for IBM-compatible PCs.

To give some insight into what intrigued me last, my copy of this book still has Post-Its on the keywords Get, Paint, and Window-- all three dealing exclusively with primitive GUI elements on the screen.  I never got anywhere with those, I'm sure of it.  But what I find funny is that the first half of the book, the user's guide, is completely devoid of marks of any kind; I doubt I ever actually read about how to actually use the development tool.  I learned by doing and didn't let myself get caught up on fundamentals like relational, logical, or functional, or string operators.  In fact, not until today did I even know that GW-BASIC even allowed low-level assembly calls!

GW-BASIC existing during a time in my life where I just didn't understand why things worked the way they did; I simply copied from a plethora of magazine articles and even a book or two (David Ahl's Basic Code Adventures and Creating Adventure Games on Your Computer by Tim Hartnell).  It was all trial and error, and it was great.

Line-numbering is obligatory.  Without line numbers, you're indicating to the IDE that you want your expression to be evaluated immediately.  Common practice was to number lines in increments of 10, with the idea that if you needed to insert another line between 10 and 20, you could choose line 12 or maybe 15 to squeeze in that additional piece of code.  A simple RENUM command will renumber the lines, including lines referenced by GOTOs.

Type-checking is kept to a minimum and is quite forgiving.  Variable types, although not required, were denoted by the following postfixes:
$ String
% Integer
! Single-precision
# Double-precision

GW-BASIC had limited support for subroutines.  Subroutines (via the GOSUB statement) were perceived as GOTOs that included a RETURN statement to bring the code execution back to where the calling GOSUB left off.

GW-BASIC's successor, QBASIC, introduced more elegant sub and function calls as well as the concept of line labels, eliminating the need to identify each line with an incremented number.  Plus it allowed mouse input, something that's easily taken for granted today.  QBASIC would set the stage for Visual Basic, which finally brought BASIC into the new graphics-rich Windows operating system.

Additional links:
http://www.atariarchives.org/bca/index.php
http://www.atariarchives.org/adventure/
http://www.gw-basic.com

Tool of the Week: Sending Faxes With HelloFax

It may not be that often that a developer needs to use a fax machine.  For some of us, having one nearby is one of the perks of working in an office building.  For others, taking a trip to the local copy shop can take a good deal of your time, cash out of your wallet, and gas out of your car.   I don't like using up more time, money, or even paper than I need.

So when I recently had to send a constantly-revised contract back and forth, I finally took a fresh look at the web-based faxing services.  It's not necessarily a new concept, scanning a document, uploading it, typing in a fax number and hitting send, but I was a bit embarrassed that I hadn't looked at these services  before spending about $20 (no joke) at FedEx while coping with two paper jams and some frustrated looks at the folks waiting to use the fax too.

The first service I tried, HelloFax, was the one I've stayed with.  I learned about it on Lifehacker and I liked the fax fact that the first few pages I sent were free.  I could upload different types of documents (PDF was my doc of choice) and they could even whip up a free cover page on the fly.  I hated creating cover pages; the Word template I created just never seemed to format right.

HelloFax didn't betray the fact I'm cheap and that I didn't use a real fax machine when it came time for me to send out some documents.  Only if you choose to use a cover page generated by the service, it'll add the phrase "Powered by HelloFax" on the page.  But otherwise, no ads.

Its interface is clean and allows you to edit your document by adding text, checkboxes, and even signatures (uploaded, drawn with your mouse, or taken as a picture from your phone and e-mailed to a special e-mail address).

Sending faxes is queue-based; when you click "send", you can walk away and the fax is sent in short order.  You can even get an e-mail (with an attached copy of what was sent) notifying you when the send is successful.

The service is free if all you want to do is send 5 pages-- which can be spread over one or more faxes)-- then it's 99 cents to send a fax up to 10 pages (then 20 cents per page after that).  If you visit their site through a referral link like this one, they'll give you 5 more free pages on sign-up.

There are pay plans, which I have not tried.  The cheapest plan, at $5 per month, offers 50 pages a month and you're given your own local phone number to receive faxes as well.  The higher-tiered plans offer more and more pages and still seem like a good deal if your life involves being chained to a fax machine.

How to Catch Multiple Exceptions in C#

Here's a technique for catching more than one exception in a try / catch block.  Typically it's best to only catch the exceptions that your code can handle, so catching the general System.Exception class is discouraged.  But in this case, we're catching the general Exception class only to determine what type of exception it is.  For any exceptions that aren't handled in the if / else if blocks, we'll re-throw the exception for the calling method to handle.

catch (Exception exception)
{
	if (exception is FileNotFoundException)
	{
		Console.WriteLine("Error: File not found");
	}
	else if (exception is ArgumentException || exception is ArgumentNullException || exception is DirectoryNotFoundException)
	{
		Console.WriteLine("Error: Path is invalid or does not exist");
	}
	else
	{
		throw;
	}
}

The entire console application (Program.cs):

// Code from www.jaypm.com
// Feel free to use, but please give credit where it's due.

using System;
using System.IO;

namespace MultipleExceptionHandler
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                using (var fileReader = new StreamReader("file.txt"))
                {
                    while (fileReader.Peek() > 0)
                    {
                        Console.WriteLine(fileReader.ReadLine());
                    }
                }
            }
            catch (Exception exception)
            {
                // Check if it's one of three different exceptions:
                if (exception is ArgumentException || exception is ArgumentNullException || exception is DirectoryNotFoundException)
                {
                    Console.WriteLine("Error: Path is invalid or does not exist");
                }
                // Otherwise check if it's another exception:
                else if (exception is FileNotFoundException)
                {
                    // NOTE: This was only an example to demonstrate handling multiple exceptions.
                    //        It's more efficient to use File.Exists(fileName) rather than using an exception
                    //        to drive your logic.
                    Console.WriteLine("Error: File not found");
                }
                // Since we caught the generic System.Exception, this "else" case will
                // throw back up anything that wasn't explicitly caught:
                else
                {
                    throw;
                }
            }
            Console.ReadLine();
        }
    }
}

"Error 1935" while installing VC80.CRT

I got the following error while trying to install the Broadcom WIDCOMM Bluetooth driver software on my Windows 7 AMD 64-bit machine:

Error 1935.An error occurred during the installation of assembly
'policy.8.0.Microsoft.VC80.CRT,type="win32-policy",version="8.0.50727.42",publicKeyToken="1fc8b3b9a1e18e3b",processorArchitecture="amd64"'.
Please refer to Help and Support for more information.
HRESULT: 0x800736FD. assembly interface: IAssemblyCacheItem, function: Commit, component: {4F6D20F0-CCE5-1492-A01F-C8B3B9A1E18E}

I was hoping that installing the same exact version of the Visual C 2005 Redistributable would do the trick:
http://www.microsoft.com/download/en/confirmation.aspx?id=21254

But although the redistributable installed without error, I still got the same "Error 1935".  A blog post (now offline, link removed) in Spanish listed the following steps:

  1. Open Registry Editor (Windows Key-R, type "regedit.exe")
  2. Navigate to:
    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control
  3. Locate a key called "RegistrySizeLimit"
    If this key does not exist, go to Edit->New->QWORD (64-Bit) Value
  4. Double-click the key and set its value to the hexadecimal "FFFFFFFF" and click OK
  5. Reboot
This didn't work for me either.  Ultimately I decided to try downloading another set of drivers, this time straight from the device manufacturer, in this case IOGEAR.  My experience with drivers straight from Broadcom had been pretty good until today, plus I knew that the USB-to-Bluetooth dongle at its heart used a Broadcom radio.

 

Installation finally went flawlessly.  The drivers were version 6.3.0.7500, a slightly lower number than the set I was fighting with, version 6.3.0.8200.

 

My reward for getting all of this working?  Through the IOGEAR GBU421, my laptop is streaming music from Pandora to my Soundfreaq Sound Platform.  Friggin' sweet, and totally worth the hassle.  Skype will even mute it whenever I'm on a phone call so I'm not searching for the remote.

 

Disclaimer
Obviously your mileage may vary, but I thought I'd share everything I did in case any or all of these steps help.  Please keep in mind that the Windows Registry is for advanced users only and using it incorrectly can result in an overpriced paperweight non-functional computer.

IDisposable Interface and Managed Resources

Very scarcely I'll see a developer disposing an object and I'll admit that I'm one of those developers who doesn't do this enough.  We're all aware of the garbage collector that runs at indeterminate times to clean up the mess we've left behind, so who cares whether we actually call an object's Dispose() method?  Why not just wait until the garbage collector does its thing?

Because the garbage collector won't free up unmanaged resources in your code.  These are files, handles, and streams.  These objects implement the IDisposable interface, which has only one method, Dispose().

In C#, there are two techniques for disposing of an object that implements the IDisposable interface:

  • In a try / finallyblock:
    var memoryStream = new MemoryStream();
    var bytes = new byte[128];
    try
    {
    	memoryStream.Read(bytes, 0, bytes.Length);
    }
    finally
    {
    	memoryStream.Dispose();
    } 
  • In a using statement:
    var memoryStream = new MemoryStream();
    var bytes = new byte[128];
    using (memoryStream)
    {
    	memoryStream.Read(bytes, 0, bytes.Length);
    } 

In the case of a using statement, the Dispose() method is implied at the end and is guaranteed to run even if an exception is hit; it's a fancy way of implementing the try / finally block.

In addition, if any of your classes use unmanaged resources, you'll want to implement IDisposable too.  Here's an example.  Please feel free to let me know if you see any issues with this implementation.

public class SpecializedMemoryStream : IDisposable
{
	private MemoryStream _memoryStream;
	private bool _isDisposed;

	/// <summary>
	/// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
	/// </summary>
	public void Dispose()
	{
		Dispose(true);

		// Tell the garbage collector that the finalize destructor doesn't need to be called for this object
		GC.SuppressFinalize(this);
	}

	/// <summary>
	/// The finalize destructor
	/// </summary>
	~SpecializedMemoryStream()
	{
		Dispose(false);
	}

	protected virtual void Dispose(bool isDisposingManagedResources)
	{
		// Check if this method was previously called:
		if (_isDisposed)
		{
			return;
		}

		if (isDisposingManagedResources)
		{
			// Free up managed resource:
			if (_memoryStream != null)
			{
				_memoryStream.Dispose();
				_memoryStream = null;
			}
		}

		// Dispose any native resources here.

		_isDisposed = true;
	}
}

It's definitely good practice to be on the eye out for objects that implement the IDisposable interface; although it takes a bit more code to implement correctly, it frees up resources that otherwise wouldn't have been reclaimed until your application closes and might help to prevent memory leaks along the way.

References:
http://msdn.microsoft.com/en-us/library/system.idisposable.aspx
http://msdn.microsoft.com/en-US/library/ms244737(v=VS.80).aspx 

Mutex Object, Explained

Suppose you have two threads running simultaneously and they are both using a single instance of a particular object (say a global variable, for example) .  How do you guarantee that only one thread uses that object at any given time?

A mutual exclusion ("mutex") is a mechanism that acts as a flag to prevent two threads from performing one or more actions simultaneously.  The entire action that you want to run exclusively is called a critical section or protected section.

Modern programming languages support this natively.  In C#, it's as simple as:

  1. Instantiating a new static Mutex object that's accessible from each thread
  2. Wrapping whatever code you want to be executed in the critical section with that object's WaitOne() and ReleaseMutex() methods in each thread
An overloaded WaitOne() method also accepts a TimeSpan object, useful if the thread should wait only until a period of time has lapsed.  Typically this is employed when there's a risk of deadlock, where two or more threads are waiting for the same mutex to become available simultaneously.  Deadlock is as bad as it sounds, since it can result in an application that has "stalled out" for all intents and purposes.

Here's a sample .NET 4.0 console application that demonstrates the getting and setting of a static global variable between two threads, first without using the Mutex object and then with it.  In the console output, one thread looped 50 times and one ran 200 times (at faster rate).  Each time it retrieved a value from the global variable that it wasn't expecting, the number of "clashes" was incremented.

Obviously this exercise was written to amplify this disadvantage, but you'll note that there is a large amount of unpredictability (clashes) when not using a Mutex object.  However when wrapping the getting and setting the global variable around a critical section, you'll note that no clashes are observed.  In theory we are introducing a small delay when one thread waits until the other leaves its critical section, but provided that you keep your critical section no larger than is absolutely necessary, slightly slower processing is a small price to pay for the benefit of having a predicable result.

Program.cs:

// Code from www.jaypm.com
// Feel free to use, but please give credit where it's due.

using System;
using System.Threading;

namespace MutexExample
{
    class Program
    {
        private static string _globalVariable = string.Empty;
        private static readonly Mutex GlobalVariableMutex = new Mutex();

        static void Main(string[] args)
        {
            Console.WriteLine("Press Enter to start");
            Console.ReadLine();
            Console.WriteLine("NOT using mutex:");

            var parametersA = new ThreadParameters { Name = "A", SleepTime = 5, UseMutex = false, LoopCount = 50 };
            var parametersB = new ThreadParameters { Name = "B", SleepTime = 3, UseMutex = false, LoopCount = 200 };

            var threadA = new Thread(GetSetGlobalVariable);
            var threadB = new Thread(GetSetGlobalVariable);

            threadA.Start(parametersA);
            threadB.Start(parametersB);

            // Wait until both threads finish:
            threadA.Join();
            threadB.Join();

            Console.WriteLine(Environment.NewLine + "Using mutex:");

            var parametersC = new ThreadParameters { Name = "C", SleepTime = 5, UseMutex = true, LoopCount = 50 };
            var parametersD = new ThreadParameters { Name = "D", SleepTime = 3, UseMutex = true, LoopCount = 200 };

            var threadC = new Thread(GetSetGlobalVariable);
            var threadD = new Thread(GetSetGlobalVariable);

            threadC.Start(parametersC);
            threadD.Start(parametersD);

            // Wait until both threads finish:
            threadC.Join();
            threadD.Join();

            Console.WriteLine(Environment.NewLine + "Press Enter to exit");
            Console.ReadLine();
        }

        private static void GetSetGlobalVariable(Object obj)
        {
            var threadParams = (ThreadParameters) obj;
            var numClashes = 0;  // Count the number of times the global variable wasn't the value it was expected

            for (var i = 0; i < threadParams.LoopCount; i++)
            {
                // Create a new value for the global variable, using a Guid object to ensure a unique value:
                var expectedValue = string.Format("Thread {0} - {1}", threadParams.Name, Guid.NewGuid());

                // If using a Mutex:
                if (threadParams.UseMutex)
                {
                    // Start the critical area / protected section:
                    GlobalVariableMutex.WaitOne();
                }

                // Set the global variable:
                _globalVariable = expectedValue;

                // Insert an artificial wait, simulating additional processing:
                Thread.Sleep(threadParams.SleepTime);

                // Get the global variable:
                var actualValue = _globalVariable;

                // If using a Mutex:
                if (threadParams.UseMutex)
                {
                    // End the critical area / protected section:
                    GlobalVariableMutex.ReleaseMutex();
                }

                // Compare the global variable with the expected value:
                var isClash = expectedValue != actualValue;
                if (isClash)
                {
                    numClashes++;
                }

                //Console.WriteLine(string.Format("Expecting: {0}{3}Actual: {1}{3}Is a clash: {2}{3}", expectedValue, actualValue, isClash, Environment.NewLine));
            }
            Console.WriteLine(String.Format("Number of Thread {0} clashes: {1} out of {2}", threadParams.Name, numClashes, threadParams.LoopCount));
        }
    }
}

Here's a ThreadParameters class that's used for storing some parameters passed to each of the threads.  It's pretty application-specific.

ThreadParameters.cs:

// Code from www.jaypm.com
// Feel free to use, but please give credit where it's due.

namespace MutexExample
{
    class ThreadParameters
    {
        public string Name { get; set; }
        public bool UseMutex { get; set; }
        public int SleepTime { get; set; }
        public int LoopCount { get; set; }
    }
}

References:
http://en.wikipedia.org/wiki/Mutex
http://msdn.microsoft.com/en-us/library/system.threading.mutex.aspx

.NET Micro Framework, the Netduino, MP3s and File Reading

Microcontrollers have truly advanced to the point where it's possible to control them through high-level programming languages.  After surviving my assembly class in college (using a MIPS processor simulator), I swore off bit-shifting and jump statements altogether and never looked back.

Had it not been for the .NET Micro Framework, I don't think I'd even consider trying to get a tiny device to do anything a bigger computer could do; the learning curve was just too high.  Sure, the Arduino prototyping platform paved the way toward where we are now, but I liked the fact that I could write code in a familiar IDE with a familiar language (C#) and get a tiny device to do my bidding-- at least blink-- in a matter of minutes.

Sure, there are still new concepts for object-oriented developers to tackle, like interrupt ports and the need to pick up hardware like resistors, breadboards, and "shields" (devices that can be plugged into the top of the device), but it can bring familiar object-oriented concepts and even multithreading to an entirely new platform. It was actually fun to pick up a soldering iron again, something I haven't done in at least 15 years.

I decided to try the Netduino Plus, a very affordable device with a tiny amount of memory and a growing number of shields that are supported.  Here's my extremely minor contribution to this effort in the post, "compatible shields and accessories."

I've piled on two shields, one for SD card reading / writing and one for playing MP3s.  Provided you're only interested in playing a song encoded at about 30kbps, it's a blast.

Here's the code for Program.cs, which plays each MP3 file on an SD card in succession on a Netduino (predates the Netduino Plus with onboard SD card):

using System;
using Microsoft.SPOT;
using SecretLabs.NETMF.Hardware.Netduino;
using SecretLabs.NETMF.IO;
using System.IO;

namespace NetduinoMp3 {
    public class Program
    {
        private static readonly int FileBufferSize = Vs1053.BufferSize * 10;

        public static void Main()
        {
            Debug.Print(DateTime.Now.ToString());
            StorageDevice.MountSD("SD1", SPI_Devices.SPI1, Pins.GPIO_PIN_D10);

            string[] directories = Directory.GetDirectories(@"\");
            Debug.Print("directory count: " + directories.Length);

            for (var i = 0; i < directories.Length; i++) {
                Debug.Print("directory: " + directories[i]);
            }

            var files = Directory.GetFiles(@"\SD1");
            Debug.Print("file count: " + files.Length);

            Vs1053.Initialize();
            Vs1053.SetVolume(0xFEFE);

            for (var i = 0; i < files.Length; i++) {
                Debug.Print("filename: " + files[i]);
                var fileStream = new FileStream(files[i], FileMode.Open, FileAccess.Read, FileShare.None, _
                        FileBufferSize);

                Vs1053.SendData(fileStream);

                fileStream.Close();
            }
        }
    }
}

And the class "Vs1053.cs", used to control the MP3 shield:

using System;
using System.IO;
using Microsoft.SPOT.Hardware;
using SecretLabs.NETMF.Hardware.Netduino;
using System.Threading;

namespace NetduinoMp3 {
    public static class Vs1053 {
        // GPIO ports:
        static private OutputPort _reset;
        static private InterruptPort _dreq;

        const Cpu.Pin PinBsync = Pins.GPIO_PIN_D2;
        const Cpu.Pin PinDreq = Pins.GPIO_PIN_D3;
        const Cpu.Pin PinReset = Pins.GPIO_PIN_D6;  // NOTE: This doesn't map to an actual pin
        const Cpu.Pin PinCs = Pins.GPIO_PIN_D9;

        // Define SPI Configuration for VS1053 MP3 decoder:
        static private readonly SPI.Configuration DataConfig = new SPI.Configuration(PinBsync, false, _ 
                0, 0, false, true, 3000, SPI.SPI_module.SPI1);
        static private readonly SPI.Configuration CmdConfig = new SPI.Configuration(PinCs, false, _ 
                0, 0, false, true, 3000, SPI.SPI_module.SPI1);
        static private SPI _spi;

        // Registers:
        const int RegisterSciMode = 0x00;
        const int RegisterSciVol = 0x0B;
        const int RegisterSciClockf = 0x03;

        public static readonly int BufferSize = 96;
        private const ushort SciMode = 0x880;  // SM_SDINEW (default) + SM_EARSPEAKER_HI

        static private bool _isInitialized;
        static private readonly byte[] ReadBuffer = new byte[BufferSize];
        static private readonly byte[] CmdBuffer = new byte[4];

        static private readonly AutoResetEvent AutoResetEvent = new AutoResetEvent(false);

        public static void Initialize() {
            if (_isInitialized)
                Shutdown();

            _spi = new SPI(CmdConfig);
            _reset = new OutputPort(PinReset, true);
            _dreq = new InterruptPort(PinDreq, false, Port.ResistorMode.PullUp, Port.InterruptMode.InterruptEdgeBoth);
            _dreq.OnInterrupt += _dreq_OnInterrupt;

            _isInitialized = true;

            Reset();

            CommandWrite(RegisterSciMode, SciMode | (1 << 2));
            CommandWrite(RegisterSciClockf, 7 << 13);
            CommandWrite(RegisterSciVol, 0x2424);

            _spi.Config = DataConfig;
        }

        private static void _dreq_OnInterrupt(uint port, uint state, DateTime time) {
            if (state == 0)
            {
                AutoResetEvent.Set();
            }
            else
            {
                AutoResetEvent.WaitOne();
            }
            _dreq.ClearInterrupt();
        }

        private static void Reset() {
            _reset.Write(false);
            Thread.Sleep(1);
            _reset.Write(true);
            Thread.Sleep(1);
        }

        static private void CommandWrite(byte address, ushort data) {
            CmdBuffer[0] = 0x02;
            CmdBuffer[1] = address;
            CmdBuffer[2] = (byte)(data >> 8);
            CmdBuffer[3] = (byte)data;

            _spi.Write(CmdBuffer);
        }

        static private ushort CommandRead(byte address) {
            CmdBuffer[0] = 0x03;
            CmdBuffer[1] = address;
            CmdBuffer[2] = 0;
            CmdBuffer[3] = 0;

            _spi.WriteRead(CmdBuffer, CmdBuffer, 2);

            ushort command = CmdBuffer[0];
            command <<= 8;
            command += CmdBuffer[1];

            return command;
        }

        public static void SetVolumePercent(int volume) {
            if (volume < 0 || volume > 100)
                throw new ArgumentOutOfRangeException("volume");

            SetVolumePercent(volume, volume);
        }

        public static void SetVolumePercent(int leftChannel, int rightChannel) {
            if (leftChannel < 0 || leftChannel > 100)
                throw new ArgumentOutOfRangeException("leftChannel");
            if (rightChannel < 0 || rightChannel > 100)
                throw new ArgumentOutOfRangeException("rightChannel");

            // TODO: Invert decibel value, divide by percent, call SetVolume(ushort leftChannel, ushort rightChannel)
        }

        public static void SetVolume(ushort bothChannels) {
            CommandWrite(RegisterSciVol, bothChannels);  // TODO: This doesn't work outside the Initialize() method
        }

        public static void SetVolume(ushort leftChannel, ushort rightChannel)
        {
            SetVolume((ushort) (leftChannel*256 + rightChannel)); // TODO: Verify no loss of fidelity
        }

        public static void SendData(FileStream fileStream)
        {
            var size = fileStream.Length - fileStream.Length % BufferSize;
            for (var i = 0; i < size; i += BufferSize) {
                fileStream.Read(ReadBuffer, 0, BufferSize);

                _spi.Write(ReadBuffer);
            }
        }

        public static void Shutdown() {
            if (!_isInitialized) return;

            Reset();

            _spi.Dispose();
            _reset.Dispose();
            _dreq.Dispose();

            _isInitialized = false;
        }
    }
}

I think there's a lot to be desired, but it's hopefully at least a good starting point for somebody and at least extends on my initial post to the Netduino forum that simply stated that the MP3 shield "worked."

Additional References:

So here I start.

It's been years since I've tried to create a new site.  I've wanted to start this one several years ago, but I've always wondered whether I really had anything new or valuable to share.  Plus I'll admit I've been only a spectator in new web development practices since I made the transition from static HTML to PHP scripting back in the late 90s.

Web development has truly grown up since I began memorizing that the <head> tag goes before <body> and that <table> tags are always followed by <tr> tags and that you can't just jump right into <td>.  Point is, it's not so much about needing to know pure HTML anymore (sorry, I'm not going to count the then-popular Microsoft FrontPage); it's all about themes, plugins, posts, and feeds.  "Search engine optimization..."  What?

It finally took a rainy day and inspiration from a modern blogger and friend of an extremely successful craft-making site, Craftaholics Anonymous, to make me question why I didn't just jump in and put into practice the concepts I've only read about for the past few years and share a bit of what I've learned.  There are a lot of better developers out there.  But if I keep waiting to find myself on the inside track of a bleeding-edge technology, then this site will remain just a thought in the back of my mind and it will never be realized.

So here I start.  I'm realizing it's about the journey of always striving to write better code, learning new techniques, and making mistakes along the way.  I'm not going to be perfect, but I'll give it my best shot.

- Jay