Implementing Dynamically Loadable Image Postprocessing Plugins

The plugins will be provided in various programming languages to show how language interoperability works in a .NET application.

8.5.1 Late Binding and Reflection

The .NET Framework provides the concept of reflection to query for information on types and metadata from an assembly's manifest, which contains information that describes the assembly completely. For a complete description of the assembly manifest, refer to the MSDN help files.

The capability of querying data from the manifest is built into the System.Reflection namespace, which defines many classes to provide the developer with a wide range of functionality. The functionality ranges from querying information about assemblies to emitting code to create types at run time.

The feature we are looking for is runtime invocation, also referred to as late binding. Late binding enables the photo editor application to query type information and to invoke methods, defined in other assemblies, at run time. The photo editor application uses reflection and late binding to load the plugin assembly functionalities at run time. The advantage of this feature is that it allows customers to write their own implementations, which can simply be exchanged with the existing assemblies in the application directory without the need to recompile the application. As a result, customers can change the behavior of the application or provide optimized versions of the plugins without any application changes or compilation.

Loading an Assembly at Run Time

In this section, we develop the base functionality to dynamically load various components in the photo editor. In the first step of the implementation we introduce a new class named PlugInInterface to the Photo Editor Application project. This new class keeps the interface definition of the dynamically loadable components separate from the application code.

First, we add some namespaces to the file. The dynamic loading uses the System.Reflection namespace to receive information on the methods defined in an assembly. Therefore, we add this namespace to the file. In addition, we use the System.Drawing namespace, which includes the Bitmap definition. To search a directory for files, we also include the System.IO namespace, and, last but not least, we use the exception management block. Therefore, we also add the Microsoft.ApplicationBlocks.ExceptionManagement to our list of namespaces.

In addition, we define some private class properties. For example, we define a private string property that holds the postfix "PlugIn.dll" of the components that we are looking for. To store the prefix of the component, we also define a string property. In addition, we define three integer values that correspond to the parameters that are needed by the image-processing calculation within the components. The resulting code can be seen in Listing 8.1.

Listing 8.1 PlugInInterface, Part 1

using System; using System.Reflection; using System.Drawing; using System.IO; using Microsoft.ApplicationBlocks.ExceptionManagement; namespace Photo_Editor_Application { ///

/// Interface used to load the provided plugins /// dynamically. ///

public class PlugInInterface { // postfix for all dynamically loadable components private static string searchPlugInString = "PlugIn.dll"; Assembly assembly; private Bitmap tmpBitmap; // values of the parameters private int param3; private int param2; private int param1; // name of the operation and prefix of the component private string sliderName; ///

/// Constructor ///

public PlugInInterface() { // // TODO: Add constructor logic here // }

Now that the properties are defined, we start to implement the actual interface that is responsible for loading a component at run time in order to execute a requested image-processing operation. We define the actual interface method, ApplyImageProcessing, as public so that it can be called by the application. It does not accept any parameters. In this method the first action we implement is to create the name of the component that we want to load. The name is a concatenation of sliderName and searchPlugInString, and the result is a string of the form "xxxxPlugIn.dll", where xxxx is the name of the requested operation.

Now that we have the complete name of the component, we search the current directory for all files with the same name. The matches are stored in a string array named filenames. To search files in a given directory, the .NET Framework class Directory is used. This class is defined in the System.IO namespace and defines a method called GetFiles(...), which returns an array of strings in which each string represents a file in the directory searched. The components of the photo editor are stored in the application directory. This means that the application always uses the assembly found the same directory as the application.

Then we check whether a file with the requested name actually was found. If no file was found, an exception is thrown. After that, we also check whether more than one match was found; if it was, we also throw an exception to indicate an error (this is very unlikely because two files in the same directory cannot have the same name). Listing 8.2 shows the implementation of the functionality.

Listing 8.2 PlugInInterface, Part 2

* * * ///

/// Searches the PlugIns directory for a DLL with the /// specified name. If none is found then an exception /// is thrown. /// Otherwise it tries to invoke the method ProcessImage /// with the specified /// parameters. ///

public void ApplyImageProcessing() { try { #region ApplyImageProcessing Code from Chapter 8 // Concatenates the operation name with // the suffix "PlugIn.dll" string operation; operation = sliderName + searchPlugInString; string[] filenames = Directory.GetFiles(PhotoEditorForm.GetApplicationDirectory, operation); // If no PlugIn is found indicate an error by // throwing an exception. if(filenames.Length == 0) { throw(new Exception("Error, PlugIn could not be found in the specified location")); } // This should never ever happen! Two files cannot have // the same name // and reside in the same directory! // But anyway the check does not hurt either. Better safe // than sorry ;-) if(filenames.Length > 1) throw(new Exception("Error, More than one PlugIn could be found in the specified location"));

Now that the component's DLL file has been found, we are ready to load the component and to invoke the method so we can apply the image-processing operation requested by the application. To query the types defined in the located assembly, we must first load the assembly. To load the assembly we use the System.Assembly namespace, which defines various Load methods and overloads. The following example uses Assembly.LoadFrom(filename). The return value is of type Assembly and is stored in the private member variable assembly. Then we load all the defined types in the assembly. As mentioned earlier, the manifest of an assembly describes the assembly. We use the GetTypes method to extract the types defined in the loaded assembly. The call to that method returns an array that contains all such types.

Member Invocation at Run Time

Parameters that need to be passed to the method to be invoked are passed with an array of type object. This array holds the parameters that are required by the ProcessImage method. To match the signature of the ProcessImage method, we create an object array of size 4. Remember, the design specification says that the ProcessImage method is defined to take four parameters. The parameters are a Bitmap (to hold a bitmap reference to the image) and three Integer parameters to hold additional information as needed.

After setting the parameters, we search the loaded assembly for the methods it defines. .NET provides the GetMethods() method, which is used to discover methods defined in the loaded assembly. The array that is returned is of type MethodInfo. The application then iterates through this array to find a method that matches the name ProcessImage.

For error-handling purposes, we also define some helper properties. First, we create a local variable of type bool with the name methodNotFound and initialize it to true. This variable is used to check whether a method with the specified name was found in the list of methods. After that, we define a const string that holds the name of the method that is invoked. According to the design specification, the name is "ProcessImage".

The final step is to create a local variable of type object. This variable is used to hold the invoked method.

Now that the setup is complete, we use a foreach loop to iterate over the methods defined in the assembly. The loop iterates over the array of methods returned by the call to GetMethods(), as described earlier. If a method with the name "ProcessImage" is found, then an instance of the type is created using Activator.CreateInstance(theType[0]). In addition, the methodNotFound flag is set to false, indicating that a method with the provided name was found in the assembly. Next, we call theType[0].InvokeMember(***)to actually invoke the method of the plugin assembly. The following parameters are passed to the invoked method:

• The name of the member to be invoked.
• A flag indicating that a method has been invoked.
• A System.Reflection.Binder object, which in this case is null and therefore default binding is used.
• The object that defines the member that is invoked.
• An array of int parameters.

After the method is called, some error handling is done if the invocation failed. Depending on the type of error that occurred, different error messages are provided.

The resulting code can be seen in Listing 8.3.

Listing 8.3 PlugInInterface, Part 3

* * * // Load the plugin assembly that was found. assembly = Assembly.LoadFrom(filenames[0]); // Query the loaded assembly for types defined. Type[] theType = assembly.GetTypes(); // Create an object array that holds the parameters for // the method to be called. object[] param = new Object[4]; param[0] = tmpBitmap; param[1] = param1; param[2] = param2; param[3] = param3; // Get all the methods defined in the plugin assembly MethodInfo[] methods = theType[0].GetMethods(); // used to remember whether a method with name // methodName was found. If not an error will // be signaled. bool methodNotFound = true; // Defines the name of the method // that needs to be defined in each plugin // in order to invoke it. const string methodName = "ProcessImage"; object assemblyObject; // Iterate through the methods defined // in the assembly that was found and // check whether a method with the specified // name is defined in the assembly. // If so, invoke the method. foreach(MethodInfo NextMethod in methods) { #endregion if(NextMethod.Name == methodName) { // Create an instance of the type found in the plugin // assembly assemblyObject = Activator.CreateInstance(theType[0]); try { methodNotFound = false; // Invoke the member whose signature matches // void ProcessImage(Bitmap, int, int, int); theType[0].InvokeMember(methodName, BindingFlags.InvokeMethod, null, assemblyObject, param); } // Exception at this point most likely means that // none of the methods with the correct name // had the correct signature! catch(Exception e) { throw(new Exception(e.Message + " Method with the specified signature could NOT be found in PlugIn DLL. Or an exception was thrown within the invoked method!")); } } } if(methodNotFound) // If no method with the correct name was defined by the // assembly // then indicate the error to the user! throw( new Exception("Error PlugIn does NOT define a method with the name ProcessImage!")); } catch(Exception exception) { // If the invocation has failed, an // error message is displayed. ExceptionManager.Publish(exception); } } }

In addition to the code shown in Listing 8.3, we implement an accessor for the private properties we defined. The complete solution can be found on the accompanying CD in the sample solution of Chapter 8.

To support the PlugInInterface we also make changes to the PhotoEditorForm class. All we need to do is to create an instance of the class in the constructor of PhotoEditorForm by adding the following lines:

// Create an instance of the plugin interface class // in order to be able to use dynamic loading. plugInInterface = new PlugInInterface();

In addition, we define a method that in turn calls the ApplyImageProcessing method defined in PlugInInterface. Simply add the method as shown here:

///

/// This method is called when a new thumbnail is /// calculated for previewing a transformation. ///

private void applyImageProcessing() { // Call the PlugInInterface method to do the magic. plugInInterface.ApplyImageProcessing(); } private PlugInInterface plugInInterface;

This completes the framework implementation of the dynamically loadable plugin component. In the next step, we extend the application to show a thumbnail image that serves as a preview pane to show the result of a requested operation.

8.5.2 Adding a PictureBox for Previewing Image Operations

This section describes the implementation of a PictureBox that is used to provide the user with a preview of an image operation before it is applied to the whole image. The first step is to add a new tab to the photo editor application GUI. Change the name of the new tab to Image Processing. After that, drag a PictureBox from the Toolbox onto it. Position and resize the PictureBox according to the GUI requirements shown earlier in this chapter. In addition, specify the following properties of the picture box:

Then select the text tool and type the text Preview on top of the PictureBox. To show an image within the preview box, we must create a thumbnail image of the loaded image. The thumbnail image will then be displayed in the PictureBox. To implement this feature, add a private member variable of type Image called thumbImage to the PhotoEditorForm class. With the private member variable created, it is now possible to store a thumbnail image of the loaded image. To create a thumbnail, we use the GetThumbnail(...) method in the PhotoEditorForm constructor, as shown in Listing 8.4. The parameters provided to the GetThumbnail call are as follows:

• The width and height of the PictureBox
• null (for the delegate Image.GetThumbnailImageAbort of GDI+, which actually is not used but must be provided)
• IntrPtrZero (which is defined as the callback data)

PicturePreView.Image is set to thumbImage in order to display the thumbnail.

Listing 8.4 Creating and Displaying a Thumbnail Image

// Create a thumbnail image from the loaded image. // The image will be displayed in the preview of the // image-processing tab. thumbImage = PictureObject.LoadedImage.GetThumbnailImage( picturePreView.Width, picturePreView.Height, null, IntPtr.Zero); picturePreView.Image = thumbImage;

If a new image is loaded, we must update and display an updated thumbImage. Therefore, add the same functionality to the OpenFile_Click event handler of the PhotoEditorForm class. With these changes in place, the preview image is shown in the newly added picture box. Compile and run the application to see the new feature working.

The next step is to implement the image-processing components that can be loaded at run time and the GUI controls needed to provide the parameters for the requested image-processing operation.

8.5.3 Implementation of a TrackBar Control

The GUI elements used to control the image-processing functionalities are, as specified in the requirements, TrackBar controls. As an example of the implementation of such a control, we'll look at the implementation of the contrast track bar.

We start by dragging a TrackBar control from the Toolbox onto the Image Processing tab. Do the same with a groupBox. Position and adjust the groupBox so that it encloses the track bar. Update the text of the group box to Contrast, and change the properties of the TrackBar as shown here:

After the changes to the properties have been made, double-click on the TrackBar to generate an event handler for the corresponding scroll event. The scroll event is called every time the track bar is moved.

This puts everything into place for us to start implementing the scroll event handler of the track bar. For the contrast calculation, only one parameter value is needed, so we set the unneeded parameters of the plugInInterface instance to 0. In addition, we set the sliderName to "Contrast", which is the name of the calculation that is requested:

plugInInterface.Param2 = 0; plugInInterface.Param3 = 0; plugInInterface.SliderName = "Contrast";

Then set Param1 of plugInInterface to the current value of the track bar control:

plugInInterface.Param1 = trackBarContrast.Value

Next, we create a copy of the current thumbnail image. We do this so that we can revert to the original thumbnail if users move the track bar to a different position without first applying the operation to the whole image (because all operations are based on the loaded image and not on the previously calculated image shown in the preview pane). The Image class provides the Clone method, which is used to get an exact copy of the thumbnail. Create the thumbnail image clone. Then call the applyImage method, which will dynamically load ContrastPlugIn.dll. A reference to the thumbnail image is provided to the interface object as the image on which the operation is performed. The complete implementation of the scroll event handler can be seen in Listing 8.5.

Listing 8.5 The Scroll Event Handler of the Contrast Functionality

///

/// Event handler for the scroll event of /// the contrast control. Calls /// applyImageProcessing(...) ///

///Sender object. ///Event arguments. /// F:image_contrast_and_color private void trackBarContrast_Scroll(object sender, System.EventArgs e) { try { // Reset the values of parameters 2 and 3 plugInInterface.Param2 = 0; plugInInterface.Param3 = 0; // Set the name of the operation to be applied plugInInterface.SliderName = "Contrast"; plugInInterface.Param1 = trackBarContrast.Value; // Creates a copy of the thumbnail image to be displayed in // the preview pane, showing the applied changes. picturePreView.Image = (Image)thumbImage.Clone(); plugInInterface.TmpBitmap = (Bitmap)picturePreView.Image; // Call to apply the requested operation to // the thumbnail image. applyImageProcessing(); } catch(Exception exception) { ExceptionManager.Publish(exception); } }

8.5.4 The Language-Independent Plugin Implementation

After we implement the required GUI controls, the preview pane, and the dynamic loading capability, the last missing piece is the plugin component itself. The way the loader is written enables us to provide the plugin components in any .NET language. The only restriction is that the assembly defines a public method called ProcessImage, which takes a reference to a bitmap and three integer values as parameters. The components must have the suffix and type "PlugIn.dll", and they must be stored in the application directory so that the application can locate the component at load time.

In the following subsections we show how to implement dynamically loadable components in different .NET languages. The functionalities are described as they are implemented.

For some of the image-processing calculations, it is necessary first to translate the color image into a representative grayscale image. The next section shows how this is done.

Using Luminance to Translate an RGB Color into a Grayscale Image

The contrast and brightness calculations that are to be provided as plugin components are very well defined on images in grayscale formats but not on color images. Therefore, it is necessary to translate the color image into its corresponding grayscale image. This means that the three RGB color components are transformed into their corresponding gray values.

One way to do that would be to take the average of the three color values, but that would result in a grayscale image that looks distorted. The reason is that the human eye is not equally sensitive to different colors. To compensate for this difference in sensitivity, we use a formula that considers this fact when translating the image. A commonly used method for this compensation is recommendation 601 of the CCIR (Comite Consultatif International des Radio-communications), which is defined as follows:

Luminance = R*0.299 + G*0.587 + B*0.114

Luminance is the gray value corresponding to the color values of a pixel defined in RGB mode. In the remainder of this chapter this formula is used to translate a given RGB image into a grayscale image if necessary.

The C++ Contrast Plugin

The first dynamically loadable component that is implemented is the contrast plugin. We implement this component using C++. To start, we add a new project to the Photo Editor solution of type Class Library (.NET) and name it ContrastPlugIn. This creates the necessary header and .cpp files.

Before starting to implement the functionality, we change some project settings in order to save the created DLL in either the bin or the bind directory, depending on the build configuration. Select the project in Solution Explorer, and right-click on it. Select Properties. Under Configuration Properties | General | Output Directory, change the entry to ......ind for the debug build configuration, and to ......in for the release build configuration.

After that, we are ready to add the necessary methods to the header file and rename the created class Contrast.

Two methods are used for the contrast functionality. One is the public ProcessImage method, which is invoked by the application. The second is a private helper method with the name calcNewValue. Listing 8.6 shows the header file with the added methods. Note that the class uses .NET-provided garbage collection, which is defined by using the __gc flag. If no garbage collection is to be used, the flag __nogc is used instead. The rest of the file contains the definition of the methods.

Listing 8.6 The ContrastPlugIn Header File

// ContrastPlugIn.h #pragma once using namespace System; namespace ContrastPlugIn { ///

/// Class used for contrast calculation. /// Dynamically bound by the application /// at run time. ///

///F:image_contrast_and_color /// public __gc class Contrast { public: // Method invoked by the application void ProcessImage(Drawing::Bitmap& image, int param, int unused1, int unused2); private: // Helper method unsigned int calcNewValue(unsigned int colorValue, double colorFactor, unsigned int pixValue, double factor, unsigned int maxPixValue, double lookUpValue); }; }

To make this change compile, we add a reference to the System.Drawing.dll namespace.

The next task is to implement the actual functionality for the contrast calculation. First, let's examine some theory about the contrast calculation.

Using the Histogram Calculation in the Contrast Functionality

The customer provides the following explanation for the contrast functionality: The photo editor application will use a technique called histogram equalization to perform the contrast calculation. Histogram equalization means that, based on a calculated histogram, the pixel values are redistributed so that each pixel value has an equal opportunity to occur within the image.

The first step is to calculate the grayscale histogram of the image. To do that, we walk through each pixel of the image. Each RGB value is transformed into the corresponding luminance value. The histogram then counts how often each luminance value is represented in the image. Thus, the histogram is an array whose size corresponds to the number of possible gray values in an image. Each pixel with a certain luminance value adds 1 to that array element. The histogram calculation can be seen in Figure 8.7.

Figure 8.7. The Histogram Calculation

The sum of all the values in the histogram array will be equal to the number of pixels in the image. For example, imagine a binary image with two possible gray values (0 and 1) that is 10 pixels by 10 pixels, with each value (0 and 1) occurring 50 times. This would result in a histogram of an array of two fields. Each field would contain the value 50 for the number of times each value (0 or 1) was represented in the image.

After the histogram of the image is calculated, we compute a lookup table for the equalized image. A lookup table specifies an array having the same size as the histogram and is used to transform any given gray value into a new (in this case equalized) value. Each pixel value is exchanged with the value provided by the lookup table, as shown in Figure 8.8.

Figure 8.8. The Lookup Table

The computation of the lookup table values is based on the previously calculated histogram. The values are calculated by accumulating the pixels of the histogram having the same or lower values. This number is then multiplied by the number of possible gray values and divided by the total number of pixels in the image.

Each pixel value is then exchanged with the corresponding value in the lookup table. The result is a new image in which the color values are evenly distributed (equalized). The implementation can be seen in Listing 8.7.

Listing 8.7 ContrastPlugIn, Part 1

// This is the main DLL file. #include "stdafx.h" #include "ContrastPlugIn.h" using namespace Drawing; using namespace ContrastPlugIn; ///

/// Method that does the contrast calculation /// based on the provided parameters. /// This method is invoked by the application. ///

/// F:image_contrast_and_color void Contrast::ProcessImage(Bitmap& image, int param, int unused1, int unused2) { // Image dimensions. unsigned int maxX = image.Width; unsigned int maxY = image.Height; // Pixel count initialized with 0. unsigned long totNumPixels = 0; // Number of possible color values. unsigned int maxPixValue = 0; // Stores color value of pixel. Color mycolor; // stores the new color values. int red, blue, green = 0; // Slider value normalized. const double step = ((double)param)/100.0; // Correction factor. double factor = (1.0 + step); int sum = 0; int pixVal = 0; double lutVal = 0; // Constant values for luminance calculation. const double rFactor = 0.299; const double gFactor = 0.587; const double bFactor = 0.114; // Only formats that have // R, G, and B values stored as // 8 bits per pixel are supported. switch(image.PixelFormat) { case(Imaging::PixelFormat::Format24bppRgb): case(Imaging::PixelFormat::Format32bppArgb): case(Imaging::PixelFormat::Format32bppRgb): case(Imaging::PixelFormat::Format32bppPArgb): maxPixValue = 255; break; default: throw(new Exception("Error, Pixel format not supported for this operation!")); break; }

Note that one of the problems with this algorithm is that it is sensitive to noise within the image.

Armed with this knowledge about histogram calculations, you'll find that the implementation of the contrast improvement functionality is relatively straightforward.

The Implementation of the Contrast Plugin

First, add two using statements to the .cpp file. In addition, add the necessary local variables that are needed for the contrast calculation.

The implementation presented in this section works correctly only when all three color values are stored as 8-bit values. To make sure that the image passed to the ProcessImage method is in the supported format, add a switch statement that checks this assumption. If the pixel format is supported, we set the maximum pixel value to 255; otherwise, we throw an exception that is caught by the application.

Next, declare the histogram as an array of unsigned integers with 255 entries (the maximum number of color values a pixel can have). Initialize all histogram values to 0. Then calculate the histogram by walking through all pixels in the image, calculate the corresponding grayscale value, and count the number of pixels having each grayscale value.

To extract a pixel's color information from an image, .NET provides a method on the Bitmaps class called GetPixel(xPos, yPos). This method returns a value of type color that represents the color values of the pixel at the x position (xPos) and y position (yPos) of the image. Use that method to get the color information of the current pixel, and calculate the luminance of the current pixel. Add the pixel to the corresponding bucket of the histogram, and increase the total number of pixels counted so far by 1.

After the calculation of the histogram is finished, the lookup table is calculated. Create a lookup table using a for loop. Iterate through all the elements of the histogram. Implement the inner for loop, which accumulates the number of pixels having gray values less than the current one. Then calculate the lookup table entry for the current histogram entry by multiplying the sum by the number of gray values a pixel can take. Divide the result by the total number of pixels in the image, and reset the sum to 0 for the next iteration (see Listing 8.8).

Listing 8.8 ContrastPlugIn, Part 2

// Define the histogram and the lookup table unsigned int* histogram = new unsigned int[maxPixValue]; unsigned int* lut = new unsigned int[maxPixValue]; // Initialize all values in the // histogram to 0. for( unsigned int q = 0; q <= maxPixValue; ++q) { histogram[q] = 0; } // Walk through the whole image and // calculate the histogram. for(unsigned int i = 0; i < maxY; ++i) { for(unsigned int j = 0; j < maxX; ++j) { // Get the color pixel value at the current position. Color mycolor = image.GetPixel(j,i); // Calculate the luminance value pixVal = (int)(((double)mycolor.R * rFactor) + ((double)mycolor.G * gFactor) + ((double)mycolor.B * bFactor)); // Add the pixel to the histogram histogram[pixVal] += 1; // Count number of pixels in // the histogram. totNumPixels++; } } // Walk through all the values in the histogram for( unsigned int k = 0; k <= maxPixValue; ++k) { for( unsigned int l = 0; l < k; ++l) { // Add all pixels that were // accounted for already sum += histogram[l]; } // Calculate the lookup table. // Accumulate all pixels with a // lower value * maximum value // divided by the total number of pixels in the // image, resulting in the new pixel value of the current // gray value. lut[k] = (int)(((double)sum*(double)maxPixValue)/ (double)totNumPixels); // reset the sum. sum = 0; }

With the lookup table in place, we can then process the image, as shown in Listing 8.9. Create a loop that walks through the entire image. Retrieve the color value for the current pixel, and calculate the luminance value for it. Look up the value for the current gray value by retrieving it from the lookup table. Then calculate each color component for the current pixel using a helper function called calcNewValue.

This helper function calculates the new color value, taking into account the contribution of the current color to the luminance factor. The result is then multiplied by correctionFactor, which represents the factor based on the track bar position in the GUI. Implement this helper function.

After the new color components of the current pixel have been calculated, set the color of the current pixel to the new color using SetPixel(xPos, yPos, Color::FromArgb(red, green, blue)). The overloaded method used to set the new color value is Color::FromArgb, which takes the three color values red, green, and blue as parameters. After the last pixel is calculated, the image is displayed on the screen.

Listing 8.9 ContrastPlugIn, Part 3

// Walk through the whole image for(unsigned int i = 0; i < maxY; ++i) { for(unsigned int j = 0; j < maxX; ++j) { // Get the current pixel value mycolor = image.GetPixel(j,i); // Calculate the resulting gray value pixVal = (unsigned int)(((double)mycolor.R * rFactor) + ((double)mycolor.G * gFactor) + ((double)mycolor.B * bFactor)); // Get the lookup table value for the current pixel. lutVal = (double)lut[pixVal]; // Calculate the new red, green, and blue value. red = calcNewValue(mycolor.R, rFactor, pixVal, factor, maxPixValue, lutVal); green = calcNewValue(mycolor.G, gFactor, pixVal, factor, maxPixValue, lutVal); blue = calcNewValue(mycolor.B, bFactor, pixVal, factor, maxPixValue, lutVal); // Set the pixel to the new RGB value image.SetPixel(j, i, Color::FromArgb(red, green, blue)); } } } unsigned int Contrast::calcNewValue(unsigned int colorValue, double colorFactor, unsigned int pixValue, double correctionFactor, unsigned int maxPixValue, double lookUpValue) { // Calculate the new color value for the current pixel. // Take the source pixel color // distribution into account. if(pixValue < 1) pixValue = 1; double result = (((double)colorValue*lookUpValue)/ (double)pixValue) *correctionFactor; // Check for overflow result = ((result < maxPixValue) ? result : maxPixValue); // Check for underflow result = ((result < 0) ? 0 : result); return (unsigned int)result; }

It is obvious from this implementation that the image-processing calculation can take some time. If large images are loaded, the calculation will not be finished in interactive speed, and the user will have to wait for the result to be displayed on the screen. Certainly there are many opportunities to optimize this code. Some of the possible optimization improvements are shown in Chapter 10. But for now, we are happy to have the functionality in place.

In the next implementation step, we add an Apply button to the GUI to enable the user to apply the image processing over the entire image. In addition, we add a Reset button to restore the originally loaded image, in case the user wants to undo the changes.

The Apply and Reset Buttons

Drag two buttons from the Toolbox onto the Image Processing tab control, and rename them ApplyProcessingButton and IPResetButton. Double-click on the buttons to generate the click event handlers for both buttons. The ApplyButton_Click method calls the applyImageProcessing method and provides the loaded image as tmpBitmap to PlugInInterface.

Next, invalidate the control and update thumbImage to reflect the change in the Preview Image pane. The implementation is shown in Listing 8.10. The IPResetButton_click event handler simply calls the already implemented Reset button event handler and updates thumbImage.

Listing 8.10 The Apply and Reset Buttons

///

/// Applies the current image-processing /// operation to the currently loaded image. ///

private void ApplyProcessingButton_Click(object sender, System.EventArgs e) { // Store a copy of the loaded image, on which the // image processing calculation is performed. plugInInterface.TmpBitmap = (Bitmap)PictureObject.LoadedImage; // Call the plugin via the interface plugInInterface.ApplyImageProcessing(); // Invalidate the image displayed to show the changes. customScrollableControl.Invalidate(); // Update the thumbnail image thumbImage = PictureObject.LoadedImage.GetThumbnailImage( picturePreView.Width, picturePreView.Height,null, IntPtr.Zero); // Let's see the result DisplayImage(); } ///

/// Resets the image to the originally /// loaded image. ///

private void IPResetButton_Click(object sender, System.EventArgs e) { this.ResetButton_Click(sender, e); // Create a new thumbImage and display it in the preview pane. thumbImage = PictureObject.LoadedImage.GetThumbnailImage( picturePreView.Width, picturePreView.Height,null, IntPtr.Zero); picturePreView.Image = (Image)thumbImage.Clone(); }

Now that we have implemented the infrastructure, the Apply button, and the Reset button, it is easy to add new plugin functionality to the application.

The general strategy for adding new functionality through dynamically loadable components (or plugins) is to add the GUI element and create an event handler for the user action on the GUI element. In the event handler, call the applyImageProcessing method and implement the plugin. In our plugin implementation, different .NET languages work together seamlessly. No special action had to be taken to use the C++ plugin within the C# code.

The Brightness Plugin Implementation Using J#

The next requirement to be implemented is the brightness correction plugin. To show that all .NET languages work as easily together as in the plugin just shown, we show the implementation of the brightness correction feature using J#. The first task is to create the GUI control and the event handler that will call the plugin.

Compared with the contrast calculation, it's fairly trivial to implement the brightness correction. First, add to the solution a new class library project in the J# language, and call the new project BrightnessPlugIn. Then adjust the project settings to save the assemblies in the corresponding directory, as described for ContrastPlugIn.

Pass the TrackBar value as a parameter to the plugin interface instance. The brightness correction functionality adds the application-provided value to the luminance value of each pixel. We normalize the passed value by dividing it by the luminance, and then we multiply the resulting value with each color value to get the resulting color. After that, we check the new value for overflow. The implementation can be seen in Listing 8.11.

Listing 8.11 The Brightness Correction Plugin

[View full width]

package BrightnessPlugIn; import System.*; import System.Drawing.*; import System.Drawing.Imaging.*; import System.Math.*; /** * Brightness Plugin * This class is dynamically loaded by the * application. Only the parameters for * sourceImage and offset are used. */ public class BrightnessPlugIn { public BrightnessPlugIn() { } /** * This method is invoked by the * application to do the brightness correction. */ public void ProcessImage(Bitmap sourceImage, int offset, int unused1, int unused2) throws System.Exception { // Luminance factors. double rFactor = 0.299; double gFactor = 0.5874; double bFactor = 0.114; // Get image dimensions. int width = sourceImage.get_Width(); int height = sourceImage.get_Height(); // Stores the color value of a pixel. Color mycolor; int maxPixValue = 0; int pixFormat = Bitmap.GetPixelFormatSize(sourceImage.get_PixelFormat()); // Check whether the pixel storage is 8 // bits per pixel. switch(pixFormat) { case 24: case 32: maxPixValue = 255; break; default: throw(new System.Exception("Error, Pixel format not supported for this operation!")); } double luminance; // Walk through the image, calculate the // new pixel values. for(int i = 0; i < height; ++i) { for(int j = 0; j < width; ++j) { mycolor = sourceImage.GetPixel(j,i); luminance = mycolor.get_R()*rFactor + mycolor.get_G()*gFactor + mycolor.get_B()*bFactor; // Make sure luminance is bigger than 1. // If the value were much smaller, // a division by 0 exception could occur. if(System.Math.Abs(luminance) < 1) luminance = 1; // Calculate the normalized offset factor. luminance = ((double)offset + luminance)/luminance; // Calculate the new color values. int redValue = calcNewValue(mycolor.get_R(), maxPixValue, luminance); int greenValue = calcNewValue(mycolor.get_G(), maxPixValue, luminance); int blueValue = calcNewValue(mycolor.get_B(), maxPixValue, luminance); // Set the new color values. sourceImage.SetPixel(j, i, Color.FromArgb(redValue, greenValue, blueValue)); } } } /** * This method calculates the final * value and checks for overflow and * underflow. */ private int calcNewValue(int oldValue, int maxPixValue, double correctionFactor) { // Calculate the resulting color value. int result = (int)(oldValue * correctionFactor); // Check for overflow. result = ((result < maxPixValue) ? result : maxPixValue); // Check for underflow. result = ((result < 0) ? 0 : result); return result; } }

In addition, we add to the project a reference to System.Drawing.dll. Compared with the calculation of the contrast correction, the brightness correction is less computationally expensive. But still, it will take some time if the calculation is applied to a large image.

The Color Correction Plugin Using Visual Basic

The functionality of color correction takes the TrackBar value and adds it to the corresponding color component of each pixel within the image.

The Red Eye Removal Tool

The last component to be implemented provides the functionality that attempts to correct the appearance of red eyes within pictures. This plugin is implemented using the C# programming language.

If the red eye tool is selected and the mouse is moved inside the image, then a red rectangle is shown at the current position of the mouse. The size of the rectangle to be drawn is 10 pixels in width and height. Then, if the user clicks the mouse, the area within the rectangle is saved and the plugin is called. The plugin walks through each pixel within the rectangle area and reduces the red value of each pixel to 80 percent of its original value.

The first step in implementing the RedEye tool is to create a button on the Image Processing tab control. Rename the button redEyeButton, change the text to Remove Red Eye, and create an event handler for the click event. To use the red eye tool, we add a new enumeration member called RedEyeTool to the graphics tool enumeration. To select the tool when the mouse button is clicked, set toolSelected to RedEyeTool in the redEyeButton_click event handler. In addition, we add a new flag, mouseIn Control, to the PhotoEditorForm class; this flag indicates whether or not the mouse is within customScrollableControl. Therefore, we set mouseInControl to true whenever the customScrollableControl MouseEnter event handler is called, and we set mouseInControl to false when the MouseLeave event occurs.

If the mouse is within the control and the red eye tool is selected, then a red rectangle is drawn at the position of the cursor. To accommodate this behavior, we add the drawing code for the red rectangle to the customScrollableControl_MouseMove event handler. Then, to update the position of the mouse, we set Param1 to the current x position of the mouse and set Param2 to the y value if the MouseMove event is raised and the red eye tool is selected. Then we draw the loaded image, invalidate the control, and draw the rectangle with the width and height equal to 10 pixels. The pen used to draw the rectangle is red.

If the mouse is not moved within the control, then we must draw the red eye rectangle if customScrollableControl is repainted. Therefore, we add the code that draws the rectangle at the current mouse position if the paint event handler of the custom scrollable control is called.

To make the tool work, the last application change we make is to change the MouseDown event handler of customScrollableControl. Therefore, we call the applyImageProcessing method with the area of the image contained within the red rectangle as parameter.

The plugin is implemented using C#. In the Photo Editor solution, create a new C# project that is of type Class Library. Name the project RedEyePlugIn, and rename the created class and constructor RedEyePlugIn. Change the build properties of the project to output the compiled assemblies into the bind or bin directory, and define a name for the XML documentation file.

We implement the plugin itself by walking through the pixels within the rectangular area shown on the screen. For each pixel within that area, the red component is reduced to 80 percent of its original value. The implementation of this feature can be seen in Listing 8.12.

Listing 8.12 The Red Eye Correction Plugin

using System; using System.Drawing; namespace RedEyePlugIn { ///

/// RedEyePlugIn class is loaded /// at run time by the application. ///

/// F:image_special_effects public class RedEyePlugIn { ///

/// Constructor. ///

public RedEyePlugIn() { } ///

/// ProcessImage method. Invoked by the application. /// Red component is reduced to 80% to remove red eyes. ///

public void ProcessImage(Bitmap sourceImage, int xVal, int yVal, int areaVal) { Color myColor; // Walk through the area of the displayed // rectangle within the image. for(int i = yVal; i < (yVal + areaVal); ++i) { for (int j = xVal; j < (xVal + areaVal); ++j) { // Get the current pixel myColor = sourceImage.GetPixel(j,i); // Set the red value to 80% of the original value // and keep the other values. sourceImage.SetPixel(j, i, Color.FromArgb((int)(myColor.R*0.8), (int)(myColor.G), (int)(myColor.B))); } } } } }

To compile the project we also add a reference to System.Drawing.dll. The resulting application is shown in Figure 8.9.

Figure 8.9. The New Application GUI

With the RedEyePlugIn functionality implemented, all required features for this iteration are provided. The next step is to write some unit tests to make sure the functionality is working as described in the requirements.