Wednesday, December 15, 2010

Using LINQ with generics and lambda expressions to add wildcard searches to a query

Adding generic search functionality as a helper class to a project has the advantage of giving greater flexibility in querying results when using linq. Using a method that returns a type of Expression<Func<T, bool>> allows the method to be used as a predicate in linq .where() clauses as per the following:

var resultSet = from p in db.Products.Where(
GenericSearch.GetWildCardCriteria<Products>("ball*","productName"))
 select p;

Given this query, the Generic type passed through is the class of the linq table that is to be queried. "ball*" is the search criteria, which would return all results starting with 'ball', using "ball*10" would give all results starting with 'ball' and ending with '10'. Alternatively, "*ball" would give all results ending with 'ball'.

Due to linq functions being relatively limited in their ability to query the database, and being much more limited that a SQL statement, the second GetWildCardCriteria<T>() method attempts to interpret a search string in terms of wildcard searches using the linq functions available, specifically, the .StartsWith(), .EndsWith() and .Contains() methods. These methods have been called through the building of the functions using the BuildExpressionFromMethod<T>() method, which is used as the search is generic, and so needs to be built dynamically so the correct type <T> and property name are used.

The BuildExpressionFromMethod<T>() can build the given .Contains(), .EndsWith(), etc. expression from the strings sent to it, and return a lambda expression to the predicate as all the functions have the same signature x.Method("comparisonText")

A direct comparison is done if there are no wildcards, and so a lambda expression for x.Field == "ball" is generated using the BuildEqualsExpression<T>() method.

Note: The PredicateBuilder extension class is also required to run this code and can be found at: http://www.albahari.com/nutshell/predicatebuilder.aspx

The code for the search is below:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Reflection;
using System.Linq.Expressions;

namespace LINQ.Helpers
{
 
    public class GenericSearch
    { 
        // use this if no predicate is added to and directly using .Where()
        public static Expression<Func<T, bool>> GetWildCardCriteria<T>(string searchString, string propertyName)
        {
            return GetWildCardCriteria<T>(True<T>(), searchString, propertyName);
        }
                
        // use this method when predcate already exists - type is inferred from expression
        public static Expression<Func<T, bool>> GetWildCardCriteria<T>(Expression<Func<T, bool>> predicate, string searchString, string propertyName)
        {
            string[] parts = searchString.Split('*');
            if (parts.Length == 1)
            {
                predicate = predicate.And(BuildEqualsExpression<T>(propertyName, searchString));
            }
            else
            {
                //Check for predicate wild card
                if (parts[0] == "")
                {
                    if (parts.Length == 2)
                    {
                        //If on only 2 parts the end with
                        predicate = predicate.And(BuildExpressionFromMethod<T>(propertyName, "StartsWith", parts[1]));
                    }
                    else
                    {
                        //Else do a contains
                        predicate = predicate.And(BuildExpressionFromMethod<T>(propertyName, "Contains", parts[1]));
                    }
                }
                else if (parts.Length == 2 && parts[1] == "")
                {
                    //handle suffix wild card
                    predicate = predicate.And(BuildExpressionFromMethod<T>(propertyName, "StartsWith", parts[0]));
                }
                else if (parts.Length == 2 && !string.IsNullOrEmpty(parts[1]))
                {
                    //handle mid string wild card
                    predicate = predicate.And(BuildExpressionFromMethod<T>(propertyName, "StartsWith", parts[0]));
                    predicate = predicate.And(BuildExpressionFromMethod<T>(propertyName, "EndsWith", parts[1]));
                }
                //handle internal wild cards
                for (int i = 2; i < parts.Length - 1; i++)
                {
                    predicate = predicate.And(BuildExpressionFromMethod<T>(propertyName, "Contains", parts[i]));
                }
                //Check for a suffix in the more than 2 parts case
                if (parts.Length > 2 && !string.IsNullOrEmpty(parts[parts.Length - 1]))
                {
                    predicate = predicate.And(BuildExpressionFromMethod<T>(propertyName, "StartsWith", parts[parts.Length - 1]));
                }
            }
            return predicate;
        }

        // propName is name of the property, ie. if T is the linq table Product, then propName could be the field "Product_Name"
        // value is the value that propName is equal to, eg. the product 'ball' would be equivalent to x => x.Product_Name == "ball"

        private static Expression<Func<T, bool>> BuildEqualsExpression<T>(string propName, string value)
        {
            try
            {
                ParameterExpression param = Expression.Parameter(typeof(T), "x");
                MemberExpression prop = Expression.Property(param, propName);
                BinaryExpression equals = Expression.Equal(prop, Expression.Constant(value));
                Expression<Func<T, bool>> expr = Expression.Lambda<Func<T, bool>>(equals, param);

                return expr;
            }
            catch (Exception)
            {
                return x => true;
            }
        }

        // propName is same as above
        // method is the method to call on the property, eg. if method = "Contains" and prop_Name = "Product_Name",
        // and comparisonText = "ball"
        // this would be equivalent to x => x.Product_Name.Contains("ball")

        private static Expression<Func<T, bool>> BuildExpressionFromMethod<T>(string propName, string method, string comparisonText)
        {
            try
            {
                MethodInfo mi = typeof(String).GetMethod(method, new Type[] { typeof(String) });
                ParameterExpression param = Expression.Parameter(typeof(T), "x");
                MemberExpression field = Expression.PropertyOrField(param, propName);
                MethodCallExpression mce = Expression.Call(field, mi, Expression.Constant(comparisonText));
                Expression<Func<T, bool>> expr = Expression.Lambda<Func<T, bool>>(mce, param);

                return expr;
            }
            catch (Exception)
            {
                return x => true;
            }
        }

        private static Expression<Func<T, bool>> True<T>() { return f => true; }
    }
}

Friday, October 15, 2010

Removing carriage returns from a text file using a DOS batch file



Hello there. Well this is really a test post more than anything else to get me aquainted with the SyntaxHighlighter set of files for code publishing on a blog available at alexgorbatchev.com. But in doing so I thought I would give an example of how to remove CR/LF characters from a text file using only batch commands.

The only trick to this is to note that when doing string manipulations in DOS, the enabledelayedepansion option needs to be set, and the variables then need to be accessed as !VAR! instead of %VAR% when being reused in a FOR loop.

@echo off 
setlocal enabledelayedexpansion

for /f %%j in (%1) do (
  set linea=%%j 
  set line=!line!!linea!
 )
echo !line! > %1

Friday, July 16, 2010

Loading Open Source maps onto the Garmin Edge 705

So, I get this new Garmin Edge 705 unit for my bike, and have gone for a few rides with it, using the built in maps thinking this is as good as it gets before it dawns on me to read the pamphlets that came with it indicating that maps are available for download from the Garmin website.

Being one to always pursue cheaper avenues for quality software, I did a quick search for free maps online for the Garmin unit, before being lead to a set of Open Source Open Street Maps for Garmin where there is a handy list of providers of maps for uploading to the Garmin Edge 705.

However, the link to the Australian maps that would be of most use to me at Open Street Map Australia were not present, showing empty ZIP files that weren't much use (Hopefully this is fixed some time soon). For these maps, I found a weekly extract of the Australian Open Street Maps at SteveZ's site that were very useful, and were actually linked to the OSM Australia website.

Once the maps were downloaded, the following steps were performed to get the map onto the Edge 705:
  • Plug in the Garmin Edge 705 via USB to the PC
  • Unzip the downloaded ZIP file
  • The unit will come up as a Removable Storage Device in My Computer if you're on a PC (on mine it was G:)
  • Copy the unzipped .IMG file to the G:\Garmin folder (or whatever drive it comes up as, so long as it is in the \Garmin folder)
  • If you browse to the Garmin folder on the device you will notice that there is already a .IMG file called gmapbmap.img - this is the default highway map for your locality and is just a very high-level view that is pretty much useless for inner city cycling.
  • Rename the gmapbmap.img file to something else, say gmapbmap.old
  • Rename the .IMG file you have copied to the unit as gmapbmap.img
  • Unplug the unit from the computer and power it up.
  • You will now notice on startup that the default map is now an OpenStreetMap map (as per image)
If you have an SD card, the .IMG file can also be copied to a \Garmin folder on the SD card, and switched to by pressing Menu on the unit and selecting Settings - Map and selecting the map from the list.

There is also a program called Img2Gps that I found online that is intended as an interface to load the files to your unit, however, for all the stuffing around with this program it would appear all it does is copy the file to your unit, which, in my opinion, you are better copying the .IMG manually so you are confident it is placed in the correct place and named correctly.

Happy cycling!