On October 27th and 28th, the American Nuclear Society took one of KeyLimeTie's newest iPad applications, ANS Radiation, to the field. While it's not a Geiger counter (oh how cool it would be if it was), ANS Radiation is a questionnaire tool they use to educate the public about their exposure to radiation around them every day.

The app prompts you with a series of questions ranging from the type of building in which you live to how many hours a year you fly and how many medical tests you undergo. This then adds up the amount of millirems per year (1/1,000th of a rem), which is the international unit for measuring radiation.

The American Nuclear Society asked KeyLimeTie to build the app so they could better engage with guests at the USA Science & Engineering Festival, a festival created to re-invigorate the interest of our nation’s youth in science, technology, engineering and math. The festival was held October 27th and 28th in Washington, DC. The ANS staff estimated about 3,000 visitors came by the booth throughout the weekend.

The American Nuclear Society is a not-for-profit, international, scientific and educational organization to unify the professional activities of those within the nuclear field. The ANS boasts approximately 11,000 members including engineers, scientists, administrators, and educators representing 1,600 plus corporations, educational institutions, and government agencies.

Need an iPad app for your company?

If you're interested in something similar, or another application alltogether, please let us know. We're happy to discuss how you can use an iPad application to help boost your brand, increase productivity or shave cost. Give us a call at 630.598.9000, and make sure to follow KeyLimeTie on Twitter.

KeyLimeTie and our client Lava Lite were featured in the article "Is your biz app-appropriate?" in this week's Crain's Chicago Business. Crain’s author, Steve Hendershot, reached out to KeyLimeTie for a perspective on the mobile app industry, allowing us the opportunity to share our experiences and observations through some of the work we’ve done for our clients.

The entire article can be found on Crain's Website.

Besides being a gripping and entertaining app, we’re truly excited about the potential influence user-generated Lava Lamp concepts may have on future product design and development. We’re also anxious to see how well the integration with Social Media for sharing lamp designs on Facebook and Twitter will increase brand awareness with the iconic company.

Social Media

We're also working with Lava Lite on social media campaigns for their signature Lava Lamp product. You can follow @LavaLamp on Twitter and become a fan on Facebook.

Let's Work Together

If you're looking to dive deeper into mobile for your company, give us a call. We'd be happy to talk through your goals and devise a plan that uses apps, mobile web, or both to help you connect with your customers on mobile.

Get Updates from KeyLimeTie

Please follow @KeyLimeTie on Twitter or "Like" us on Facebook. We do our best to share useful information about the web, enterprise software, mobile technologies and how you can practically use social media for business.

Mozilla Firefox has a very powerful add-on tool called Firebug. Firebug allows web developers to inspect web pages to do such things as identify CSS styles being applied to sections of a page, easily inspect sections of a page, and to tweak style values to properly determine values for padding, font sizes, margins, and the like. Firebug is also extremely useful in tracking down and debugging layout issues on a page, especially trying to determine what style is actually being applied to a section of a page. Also, you can step-wise debug Javascript code being run by the browser.

To get started, first install Mozilla Firefox, if not already installed (http://www.mozilla.com/en-US/firefox/ie.html).  Then go to https://addons.mozilla.org/en-US/firefox/addon/1843 (using Firefox) to download and install Firebug. When done, you'll see the Firebug option in Mozilla's Tools menu. Selecting Tools -> Firebug -> Open Firebug will display the Firebug panes at the bottom of the browser:




Click on the Inspect button. Now you can mouse around the page and see the defined sections of the page (surrounded by blue border):




The lower pane will display the relevant section of the page being moused over. You can also click on the moused-over area, which selects the area (and takes you out of inspect mode). The right pane will show what styles are in effect for the selected area, and you can also mouse over the HTML section to highlight the section on the page, and can click on the HTML pane lines to see the relevant styles, including the hierarchy/cascade effect of a given style.

When an HTML line is selected, you can view and tweak the associate styles. This is very useful when debugging the page to determine both what style is (or isn't) being applied, and to determine correct values so the page is displayed as desired.




You can also disable a style setting to see its effect, by clicking to the left of the setting in the right pane (marked by a red 'do not' symbol):




Using combinations of these tweaking mechanisms and you can both easily see why a page displays and what the correct settings should be. You can also add new setting values in the right page.

Lastly, Firebug can be used to debug Javascript. You can set breakpoints, watch values, and step lines of Javascript in real-time:




In conclusion, Firebug helps to take out the guesswork in page layout and very easily shows what styles are being applied to sections of a page, as well as the page layout itself. For any serious web developer it is an essential tool. This blog is by no means a comprehensive explanation of all of Firebug's functionality, but rather a basic introduction of its core features. As of this writing, there is no equivalent tool for Internet Explorer. And, of course, IE and Mozilla don't always display a page the same way, typically because of the way the two browsers interpret styles.

If you need a solution that allows users to quickly change their NT account passwords on a web server that resides outside of their domain, you can use the DirectoryServices namespace. One example of when this might be necessary is in the case of a Windows SharePoint Service site that is exposed to an external set of users. The code below and attached solution shows how to do this.

You begin by adding a reference to the System.DirectoryServices.dll assembly. Next, create a web form with 2 Labels, 2 Textboxes and a Button control, like the one shown below:

In the code behind, add the following:

string userName = string.Empty;

    protectedvoid Page_Load(object sender, EventArgs e)
    {

        userName = HttpContext.Current.User.Identity.Name.ToString();
        lblLoggedInUser.Text = userName.Split('\\')[1];
        lblMessage.Text = string.Empty;
        lblMessage.ForeColor = System.Drawing.Color.Red;
    }

The first 2 lines of code in the Page_Load event get the current user’s domain and user name and writes the user name portion to the label control. Putting the user name in a label control versus a text box prevents users from changing the passwords of users. Next, add the following code and wire this event to the Button’s Click event:

protected void btnResetPassword_Click(object sender, EventArgs e)
    {
        try
        {
                            
            string user = userName.Split('\\')[1];
            DirectoryEntry AD = new DirectoryEntry("WinNT://" + Environment.MachineName + ",computer");
            DirectoryEntry NewUser = AD.Children.Find(user);

            NewUser.Invoke("SetPassword", newobject[] {txtConfirmPassword.Text});
            NewUser.CommitChanges();

            lblMessage.Text = "Password change successful.";
            lblMessage.ForeColor = System.Drawing.Color.Green;
        }
        catch (Exception err)
        {
            // set the error message
            Response.Write(err.Message);
        }
    }
}

The first three lines in the code above create an instance of the DirectoryEntry class based on the name of the computer and then query that computer for the specific user. Once the directory entry for that user is located, the Invoke method is called on the directory entry, passing in the command argument for setting the new password. This is followed by a call to the CommitChanges method. Lastly, in order to run this code on the web server, the web.config file needs to be modified so that the solution impersonates a user with privileges to update security settings:

<identity impersonate="true" userName="user" password="pwd"/>

Once the page is made available on the server, you can integrate it with SharePoint by adding a link to the SharePoint site, or by linking to it through a Page Viewer web part.

We recently built website software for a client that had a requirement to handle 1,000 concurrent users for 3 minutes. During the initial stress testing, the software performed perfectly up to 250 users. Once it hit 250 users, we started to receive exceptions and page content wasn't being displayed properly. After some investigation, KeyLimeTie senior developer, Michael Wick, determined the problem was around our use of the DataView class. He believed the code was not thread safe and causing threads to overwrite each other.

We went to MSDN to see if the class is thread safe and according to Microsoft, "This type is safe for multithreaded read operations. You must synchronize any write operations." What's the definition of "write operations"? In our case, the DataTable the DataView consumed was being written to once (when the website started up), but was being filtered many times in a static class. We felt the way we were using the class constituted as multiple "write operations". To confirm this, we created a test application.

Download Application only
Download Source Code

Code
In the code below, the DataView is initialized two completely different ways:
Line 4: A new instance of the DataView class is created (good)
Line 6: Uses a shared view of the DataTable (bad)

1//Create DataView
2DataView dv = null;
3if (useNew)
4    dv = new DataView(dt);
5else
6    dv = dt.DefaultView;
7
8//Apply filter
9dv.RowFilter = "RandomNumber = " + randomNumber.ToString();
10
11//Return num rows found
12return dv.Count;

With low usage, Line 4's code will work just fine. Once your software starts to pick up more usage, it will begin to error. Please run the test application we have provided to see for yourself. The screenshot below illustrates what happens when a new instance of the DataView class isn't created every time it's used. The feedback in the application shows the thread numbers and their operations. Notice the DataView code has been executed over each other's thread causing unexpected results. Next, check the "Use new keyword" checkbox and you'll see the expected results are always met (no errors reported).

With SQL Server 2005, you can use .NET Framework languages to create database objects and retrieve and update data. You can create stored procedures, triggers, aggregates, user-defined functions, and user-defined types, using any one of the CLR-compliant languages. In this article, I will cover the steps involved in creating and debugging a stored procedure in C#.

To begin, it's worth noting that there are several advantages to using managed code over T-SQL:

Enhanced programming model    .NET Framework languages offer constructs and capabilities previously unavailable to SQL developers. Visual Basic, Visual C#, and Visual C++ provide capabilities that are not available in Transact-SQL, such as arrays, sophisticated exception handling, and reusability of code.

Reusability of Code    A library of managed assemblies can be created and distributed more easily than a Transact-SQL script can be distributed. 

Leverage Existing Skills    You can use and enhance your skills in the languages and development environment in which you are already experienced to create database objects.

Richer developer experience    When you develop database objects using the SQL Server project template, you have complete integration with the project system, including building, debugging, and deployment to multiple servers.

Security    When you use database objects created using Visual Basic, Visual C#, or Visual C++, the code-access security of those languages is combined with the user-based permissions in SQL Server.

Creating the Database Project

The first step in creating a stored procedure (or any database object) is to create a SQL Server project in Visual Studio:

1. From the File menu, create a new project.

2. In the New Project Dialog, select and expand a language node in the Project Types area.

3. Select the Database node.

4. Select the SQL Server Project template.

5. Click OK.

Next, you will prompted for the database connection you are using. Unless this database has had CLR-integration enabled previously, you will need to run the following in SQL Server:

sp_configure 'clr enabled', 1
GO
RECONFIGURE
GO

Creating the Stored Procedure

At this point, you are ready to start creating database objects. To create a new stored procedure, do the following steps:
1. Open an existing SQL Server Project, or create a new one.  From the Project menu, select Add New Item.

2. Select Stored Procedure.

3. Type a Name for the new stored procedure.

4. Add code to run when the stored procedure is executed.

5. In the sample code below, I have created a stored procedure that accepts one parameter and queries the Product table of the Adventure Works database. The code should look very familiar, utilizing Connection, Command and Parameter objects. One object that may not be familiar is the SQLPipe. This allows managed stored procedures running in-process on a SQL Server database to return results back to the caller.

    [Microsoft.SqlServer.Server.SqlProcedure]
    publicstaticvoid SelectProductByProductID(int productID)
    {
        using (SqlConnection conn = 
new SqlConnection("context connection=true"))
        {
            try
            {
                SqlPipe pipe = null;

                SqlCommand SelectProductCommand = new SqlCommand();

                SqlParameter productIDParam = 
new SqlParameter("@ProductID", SqlDbType.Int);

                productIDParam.Value = productID;

                SelectProductCommand.Parameters.Add(productIDParam);
                SelectProductCommand.CommandText = 
"Select * from Production.Product where AProductID = @ProductID";

                conn.Open();

                SelectProductCommand.Connection = conn;

                pipe = SqlContext.Pipe;
                pipe.ExecuteAndSend(SelectProductCommand);
            }
            catch (SqlException sqlEx)
            {
                //code to handle or log the sql exception here 
                Console.WriteLine(sqlEx);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }

Once the code is ready, you can build and deploy the stored procedure to the database by pressing 'F5'. To test the deployment, there is a file in your project called test.sql. In this file, you can add script calls to the obects you just created. For example, to call the stored procedure created above, enter:

EXEC [dbo].[SelectProductByProductID] 4

That's it. If the stored procedure was written correctly, you will now see this item in list of stored procedures in Enterprise Manager. However, you'll see the object has a lock icon next to it, as shown below. This is because the object is not editable outside of the Visual Studio project.





Conclusion

At this point, you might be asking 'so where are the benefits?'. Let's look at a few:

Exception Handling: if the stored procedure returns an error, say a constraint violation for example, this error gets converted into a SqlException that you can then handle in your managed code. To try this, change the column in the stored procedure text above to AProduct and hit 'F5'. You'll see that the catch block is entered. This gives a lot of flexibility in how you handle errors that don't readily exist in T-SQL.

Deployment: If you need to deploy the objects you created to another server, you simply change the database connection in the project properties and hit 'F5'. No more creating and managing scripted stored procedures.

With all of the applications running on a server, it is important to keep their output pruned. Even with a basic server running a few websites, you can expect to have:
- IIS logs
- SQL Server database backups
- Application-specific logging

Over time, these files can consume a lot of disk space and potentially disable a server. Over the past 5 years, I have worked on-site with nine clients. Of these nine clients, seven of them have encountered this problem. In fact, my most recent client had this problem last week! It happens so often that when someone said the server was was running slow and throwing errors to the web browser, the first thing I said was "I bet the hard drive is full"...and it was.

Why does it happen so often? There are many reasons, but the main reason is because nobody is monitoring the disk space (or monitoring tools are not installed) and the files aren't being properly pruned or archived. To resolve this issue, I created the KeyLimeTie Log Cleaner application free for anyone to download and use. I purposely made it very simple to configure and install; there are plenty of features and options we could add.

To get it running:
1. Download Application
2. Unzip the files to a folder on your hard drive.
3. Open the "profiles.xml" file and edit it for your needs.

<?xml version="1.0" standalone="yes"?>
<Profiles>
	<Profile>
		<!-- Keep this application's logs pruned to 15 days -->
		<Description>KeyLimeTie Log Cleaner</Description>
		<FolderPath>D:\Projects\KeyLimeTie\KeyLimeTie.LogCleaner\Logs\</FolderPath>
		<Extension>txt</Extension>
		<DelOlderThanDays>15</DelOlderThanDays>
		<Recursive>false</Recursive>
	</Profile>
	<Profile>
		<!-- Example: Keep IIS Logs pruned to 15 days -->
		<!-- Setting Recursive true prunes all ftp and website logs -->
		<!-- As new websites are added, no need to update profiles -->
		<Description>All IIS Logs</Description>
		<FolderPath>C:\Windows\system32\LogFiles\</FolderPath>
		<Extension>log</Extension>
		<DelOlderThanDays>15</DelOlderThanDays>
		<Recursive>true</Recursive>
	</Profile>
	<Profile>
		<!-- Example: Keep SQL Server backups pruned to 1 week -->
		<!-- Setting Recursive true prunes all database backups -->
		<!-- As new databases are added, no need to update profiles -->
		<Description>All Database Backups</Description>
		<FolderPath>D:\Databases\Backups\</FolderPath>
		<Extension>bak</Extension>
		<DelOlderThanDays>7</DelOlderThanDays>
		<Recursive>true</Recursive>
	</Profile>
</Profiles>

Fields:
- Description: Profile description
- Folder Path: Location of files to be deleted
- Extension: Extension of files to be deleted
- DelOlderThanDays: Number of days to delete files older than today
- Recursive: Tells application to look for files in subdirectories

Schedule Application:
I installed the software on a few servers and simply created a Scheduled Task through Windows. I have it set to run every day at 4am (after the SQL Server backups are run).

Some ideas...
- Change "Extension" option to be "MatchPattern". Right now, the code searches for all files by the extension. You can put "*" in there and it'll prune all files, but it could be made to be more powerful.
- Create as Windows Service. I have created similar applications as Windows Services...and it's very easy to change this to a Windows Service. But I wanted this first version to be very simple and simple to install. I have seen people have issues with installing Windows Services.
- Create an interface to manage the profiles. It's so easy to update this really isn't necessary.

This blog entry is intended to illustrate the steps necessary to set up a Windows Vista machine to use Visual Studio 2003 (.NET 1.1).  Most of the information was obtained from the following blogs:

http://citruslime.blogspot.com/2007/04/visual-studio-2003-web-debugging-on.html

http://blogs.iis.net/brian-murphy-booth/archive/2007/03/09/how-to-setup-asp-net-v1-1-visual-studio-net-2003-projects-on-iis7-vista.aspx

 

Steps

1.        Setup your user account as a local admin on the Vista machine and switch off the "user account control”.

a.        Open Control Panel -> User Accounts.

b.       Select “Change your account type”.

c.        Select “Administrator” (if not already Administrator).

d.       Press “Change Account Type”.

e.       Select “Turn User Account Control on or off”.

f.         Uncheck “Use User Account Control (UAC) to help protect your computer”.

g.        Press “OK”.

2.        Ensure your user account is included in the Debugger User group.

a.        Note: It appears this is not possible in certain versions of Vista, such as Home Premium.  Click the following link for more information:   http://windowshelp.microsoft.com/Windows/en-US/Help/0faddcfc-e2a9-4297-a429-3f7e83fe6e361033.mspx .

3.        Ensure that .NET 1.1 **SP1** is properly installed.

·          Vista does not include .NET v1.1 by default.

·          Because .NET 1.1 is not included by default, .NET v1.1 *SP1* is also not included.

·          Without SP1, W3WP.exe will crash when running an appPool under v1.1 due to DEP

·          To check this, make sure that "c:\Windows\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll" is version "1.1.4322.2032" or higher.

·          Unless you are 100% sure that SP1 for .NET is installed, you *really* should double-check this.

·          If you need to install .NET 1.1 SP1, you can download the upgrade here: http://www.microsoft.com/downloads/details.aspx?familyid=A8F5654F-088E-40B2-BBDB-A83353618B38&displaylang=en.

4.        Enable IIS 6.0 compatiblity.

a.        Open "Control Panel".

b.       Double-click "Programs and Features".

c.        Expand "Internet Information Services".

d.       Click on “Turn Windows Features On and Off”.

e.       Expand "Web Management Tools".

f.         Check “IIS 6 Management Compatibility”.

5.        Enable ASP.NET Application Development Features, if not already done so.

a.        Open "Control Panel".

b.       Double-click "Programs and Features".

c.        Expand "Internet Information Services".

d.       Click on “Turn Windows Features On and Off”.

e.       Expand “World Wide Web Services”.

f.         Expand “Application Development Features”.

g.        Check “ASP.NET”.

                                                  Note:   “.NET Extensibility”, “ISAPI Extensions”, and “ISAPI Filters” are dependencies, so they become automatically checked.

6.        Register v1.1 with IIS.

a.        Open a CMD prompt.

b.       Change your directory to c:\Windows\MIcrosoft.net\Framework\v1.1.4322.

c.        Run "aspnet_regiis -ir -enable".

·    "ir" registers v1.1 with IIS but doesn't change any existing script mappings.

·    "enable" marks aspnet_isapi.dll as "Allowed" under "ISAPI and CGI Restrictions".

·    aspnet_regiis should also create a new AppPool under "Application Pools" called "ASP.NET 1.1" that is configured with the "Classic" pipline, and "Enable32BitAppOnWin64" set to true if a 64-bit OS.

7.        Make the new "ASP.NET 1.1" appPool the default.

a.        Open the IIS manager.

b.       Select the "Web Sites" folder.

c.        Under "Actions" on the upper right, click "Set Web Site Defaults...".

d.       Change the "Application Pool" setting to "ASP.NET 1.1".

8.        **Alternative step to 7** - Change the AppPool to "ASP.NET 1.1" after creating the ASP.NET project instead of making it the default

a.        Create the v1.1 ASP.NET project via Visual Studio. Attempting to run the project at this point will fail if the 1.1 appPool is not the default.

b.       Open the IIS manager.

c.        Right-click the newly create application directory and choose "Advanced Settings".

d.       Change the "Application Pool" to "ASP.NET 1.1".

e.       Go back to Visual Studio and attempt to run/debug project.

9.        Adjust ISAPI and CGI restrictions in IIS.

a.        Go into IIS 7.0 manager and select your server.

b.       Select ISAPI and CGI restrictions.

c.        Ensure ASP v1.1 has the restriction set to Allowed.

10.    Launch Visual Studio 2003 as Administrator.

11.    Create a simple web application with some event handling to test debugging (a button that displays text in a label control is what I used to test).

12.    Attach to the w3wp.exe process.

 

13.    Set breakpoints where appropriate.

14.    Launch the website in a browser (eg. http://localhost/YourWebApp/YourPage.aspx ).

15.    Ensure your breakpoints are being hit.

16.    That’s it!  Happy coding.

 

Over the years, we have built a number of Mass Emailing applications for customers (some web, some windows apps).
I took a recent version I created and added more to it and am now making it available to anyone.

Download Application

Features
 - Use email list with custom fields from database or cut-and-paste in an email list.
 - Validate email addresses and view invalid email addresses.
 - Specify all fields of an email including From, Reply To, CC, BCC, Subject and Body. Attachments coming soon!
 - Allows for custom fields in the title and body.
 - Send test email to verify formatting accuracy.

Over the past several years, we have implemented PayPal into our websites and many of our customer websites. Since PayPal created their Developer Program, the methods to access PayPal shopping carts and checkout has continued to grow. There are easy a half dozen ways to do it today [Learn More]. One popular method is the Payment Data Transfer (PDT) program. Back on 1/28/2006, I blogged about how to easily implement PDT into your website.

Problem: Throughout the life of the PDF program, several versions of the PayPal WebControls have been released. The most recent version is v1.0.22.19341. But there's a problem with the release that PayPal never resolved. When using the DLLs and .NET Web Controls to test in the Sandbox environment, the UseSandbox property is always ignored and the Postaction is always set to the Production environment. This makes it impossible to test in their Sandbox environment.

Solution
We created a very simple class to override the PostAction property. See code below.
Download code

using System;
using System.Data;
using System.Configuration;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

using PayPal.Web.Controls;

namespace PayPalCustom
{
    public class CustomUploadCompleteCartButton : UploadCompleteCartButton
    {
        public override string PostAction
        {
            get
            {
                return this.UseSandbox ?
                    "https://www.sandbox.paypal.com/cgi-bin/webscr" :
                    "https://www.paypal.com/cgi-bin/webscr";
            }
        }
    }
}

To implement into your website:
1. Add PayPalCustom.dll as a Reference in your application.
2. Change the "Register" tag at the top of your page to:
<%@ Register TagPrefix="cc1" Namespace="PayPalCustom" Assembly="PayPalCustom" %>
3. Change the UploadCompleteCartButton WebControl tag to
<cc1:CustomUploadCompleteCartButton id="btnCheckout" runat="server" OnClick="btnCheckout_Click"></cc1:CustomUploadCompleteCartButton>

I just got the new iPhone and it's amazing. What you see it doing in the commercials is accurate...and it does a lot more! So right away, I want to know how to implement my own software. Maybe start developing some 3rd party applications. I researched a little and found out no iPhone Software Development Kit (SDK) is available. It seems like Apple is instead telling people to develop applications as webpages. But then you cannot actually interact with the iPhone...and you cannot access the iPhone database (Contacts, Favorites, etc.).

I searched a bit more and found the following sample webpage released by Apple:
http://developer.apple.com/samplecode/Puzzler/

View webpage with your iPhone:
http://www.iarchitect.net/Uploads/126/index.html

Note: When trying to view the webpage in a standard browser (IE, Firefox), it doesn't display much. The weird thing is that if you click "Print Preview", you'll see the game.

Description
"Puzzler" is a fun and interactive game that illustrates the use of web standards and JavaScript for the iPhone. This application makes advanced usage of mouse-handlers for user-input.

To play the game simply double-click or double-tap on any set of 2 or more balls of the same color that are touching. The balls will disappear and any balls above or to the left of the balls you just eliminated will shift into new positions. The goal is to clear all the balls from the screen.

Screenshots
Click any image to see a larger version

Navigate to iArchitect.net	Click into this Blog	Start Game
Tap circles with finger	All cleared. You win!

At one time or another, we will all find ourselves trying to remember the sa password. Now, thanks to Rodney Landrum's article and the sp_help_revlogin stored procedure, there is an easy way to deal with this.

You can find his article here

 

www.petermorano.com

I recently had the need to validate an XML document against its schema, so I quickly threw together this application.

It includes the XSD document (schema) for the Google Sitemaps program.

To use:
1. Download the application
2. Unzip the two files to a folder on your computer.
3. Run the application and change the XSD location to where the "sitemap.xsd" file is located.
4. Enter in the path of the XML document and click "Validate".

The application will return with a success message or a list of error messages.

Version 4.1.1 is now available with some bug fixes:
 - Scheduling Service may run multiple times if spidering process takes over 1 hour.
 - FTP cannot communicate with server in some rare scenarios.
 - Edit Results vertical scrollbar sometimes becomes disabled.
 - Spidering error message "Object reference not set" occurs when spidering stopped before completing.

This is a beta release and has been tested.
New users: I recommend all visitors who have not installed my Google software to install this beta version.
Existing users: Please uninstall the current version and install this version. All websites and preferences will remain.

Download Sitemap Generator v4.1.1

I have tested the main application and it's solid. It runs very fast and all of the new features run great. The scheduling part appears to be running great, but I still want to test it out on a couple more operating systems (it's very complicated code and I want to make sure it runs perfectly). I have finished testing it on Windows XP and Windows 2003, but still want to test it on Windows 2000. I plan to complete this tomorrow and release ALL of version 4.0. But if you want to get a jump start with the new features, the download link is below. When I publish the final v4 tomorrow, it will be an installation package. And if you don't care about the scheduling feature, then the download link below will be perfect for you.

You'll notice the entire application is now branded with my company name, iArchitect. This is a decision I recently made to push the product under my company name with other products I will be releasing soon. One product coming soon, Web Analyzer, will be free for all licensed Sitemap Generator users! I'll let you know when it's available.

After a long, long wait, I finally have version 4.0 ready! A few licensed users have been using it for the past week and have identified only a couple minor glitches. I am confident the code is solid and ready to be released. I am finishing up some installation package items and will be releasing the new version Monday night (tonight). Thank you everyone for the wait. It will be well worth it! In case you're curious, here are the features that have been added:

When creating a new website profile, you can now copy the preferences from another profile Ability to specify Google sitemap file name HTML title is now extracted while spidering and displayed on the HTML sitemaps When spidering websites, the software now sets the "User Agent" property to "iArchitect Sitemap Generator v4.0" When spidering websites, the software now handles "302 Redirect" response codes When spidering is completed, all found webpage paths are sorted Updated XML header element per Google's update Customize your HTML sitemaps with header and footer templates Ability to export to RSS format Ability to copy sitemaps to a local path (useful when running software on web server) Ability to schedule application to run automatically Help tab added which displays online forums website Small advertising image area added in header of software; Licensed users can disable it Rebranded software; instead of seeing Brian Pautsch, you will now see iArchitect (my company name) Bug: When deleting a website profile, not all files were deleted Bug: For very large sites, the google sitemap index file had "/google/" in the paths

I have been using the Microsoft Enterprise Library for many years now. For the first time, I had serious problems deploying them to a production environment today. After about an hour of searching online, I found out that a lot of people were having this problem and only a few people had figured out how to resolve it. Here's what I did....very easy, very fast.

Problem:
Deployed web application with any Microsoft Enterprise Library component and received the follow error:

Server Error in '/' Application Required permissions cannot be acquired. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.Security.Policy.PolicyException: Required permissions cannot be acquired. Source Error: An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. Stack Trace: [PolicyException: Required permissions cannot be acquired.] System.Security.SecurityManager.ResolvePolicy(Evidence evidence, PermissionSet reqdPset, PermissionSet optPset, PermissionSet denyPset, PermissionSet& denied, Boolean checkExecutionPermission) +2736869 System.Security.SecurityManager.ResolvePolicy(Evidence evidence, PermissionSet reqdPset, PermissionSet optPset, PermissionSet denyPset, PermissionSet& denied, Int32& securitySpecialFlags, Boolean checkExecutionPermission) +57 blah blah blah...

Explanation:
Simply put, the Microsoft Enterprise Library is trying to perform operations that it is not allowed based on the trust level set on that server.
I have never received this error before since I host all of my code on my own dedicated servers (where I have a "Full" trust set).
Recently, I have been opening accounts for customers at WebHost4Life.com (I'm sort of trying to get out of the hosting business due to the liabilities). WebHost4Life must also have the trust level set to "Full" since I've never had problems there. Just today, I deployed code to a new hosting provider. I believe they have their trust level set to "Medium" and this is why I'm getting the error.

Resolution:
1. A few months ago, Microsoft released updated code to resolve this issue.
Click here to download the code updates

2. For any application block to read information from configuration files, it is necessary to grant the application ConfigurationPermission (which is not provided by default in medium trust). You can add the requirePermission="false" attribute to the application's configuration section definitions. For example:

<?xml version="1.0"?>
  <configuration>
    <configSections>
    <section name="dataConfiguration" requirePermission="false"
       type="Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSettings,
             Microsoft.Practices.EnterpriseLibrary.Data"/>
    </configSections>
	.
	.
	.
</configuration>

That's it...pretty easy and fast.

Recently, I've been using XML a lot in my side projects. When binding XML to GridViews, Repeaters and DataLists, it's very easy to set the element to display with an XPath command. For example, here's how to bind to DataList:

1<asp:DataList ID="dlBooks" runat="server">
2    <ItemTemplate>
3        <%# XPath("Title") %><br />
4        by <%# XPath("Author") %><br />
5        <br />
6        <i><%# XPath("BookText") %></i>
7    </ItemTemplate>
8</asp:DataList>
9
10
11protected void BindXML(string strXML)
12{
13    eNomInterface objAPI = null;
14    XmlDataSource xds = null;
15    try
16    {
17        xds = new XmlDataSource();
18        xds.Data = objAPI.Results.InnerXml;
19        xds.XPath = "*/Books";
20        dlBooks.DataSource = xds;
21        dlBooks.DataBind();
22    }
23    catch (Exception ex)
24    {
25        //Log exception
26    }
27    finally
28    {
29        if (xds != null)
30        {
31            xds.Dispose();
32            xds = null;
33        }
34    }
35}

But as I mentioned in a recent blog, I haven't memorized all of the XPath expressions. I know there aren't a ton, but who has the time. When I'm writing code, I often reference other code to find the expression I need (what a pain to search through code). Then when I test it out, I'm constantly reworking the expressions until I get what I want. So that's when I decided to write this little application.

This application basically allows you to select or paste in source XML. Then you can enter in an XPath expression and select the command to execute. After doing so, the results (or error) will display. You can also take your results and load them into the source XML textbox with a button click...this allows you to easily test XPath expresssions on results from previous queries. This is much better than experimenting in your code!

The XPath expression examples all came from MSDN. The application has links to the website and I also supplied the example XML and XPath expressions. Also, I pulled in all of the MSDN examples so you can easily find an example of the expression you're looking to execute.

Download Application
Download Source code

The Midwest Scuba Diving website I recently created in ASP.NET 2.0 needed the URL Rewriter logic implemented. So I took about 15 minutes and converted the ASP.NET 1.1 version to 2.0. This is the database-capable version.

What's different? Mostly .NET differences (partial pages, web.config, etc.), but I also upgraded the Data Access logic to use the 2.0 version of the Microsoft Applicaton Blocks.

And be sure to check out the scuba site:
http://www.midwestscubadiving.com

Download URL Rewriter for ASP.NET 2.0

To learn more about the URL Rewriter or to get the Database for the application, check out the 1.1 version write up: http://www.brianpautsch.com/ShowItem48.aspx

What's URL Rewriting?
About 2 years ago, a customer asked me about implementing URL Rewriting into their website. I had never heard of such a thing, so I told them I’d look into it. In case you’ve never heard of it, here’s the basic explanation:

All webpages that display dynamic text usually have querystring values passed into them so they know what to display. The website you’re on now, brianpautsch.com is a good example of it.

For example, http://www.brianpautsch.com/BlogEntries.aspx?Y=2006&M=03 would tell the webpage to display all items from March, 2006. Try it out.

Now, go to http://www.brianpautsch.com/Blog/2006/3/ - Same results, huh?

Here’s what happened. Behind the scenes, I added a rewriter rule to say: change all URLs that are “ShowItems” + 4 numbers + 2 numbers + “.aspx” to “ShowItems.aspx?Year=” + 4 numbers + “&M=” + 2 numbers.

That’s it…sounds easy enough, right? Not really.

Fortunately, I came across an article by Scott Mitchell on MSDN:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnaspp/html/urlrewriting.asp.
So first off, understand that I did not write this code. I simply extended it to allow you to store your rules in the web.config or in a database. The web.config is a great location if the rules rarely change and only use regular expressions. In my case, I’ve had 6 or 7 customers who have new rules added every day. They basically create a rule for every article, blog entry, etc. they publish.

Why should I implement URL Rewriting?

• Some public search engines and most site and intranet search engines will index URLs with dynamic URLS, but others will not. And because it's difficult to link to these pages, they will be penalized by engines which use link analysis to improve relevance ranking, such as Google's PageRank algorithm. In Summer 2003, Google had very few dynamic URL pages in the first 10 pages of results for test searches.
• Search engine robot writers are concerned about their robot programs getting lost on web sites with infinite listings. Search engine developers call these "spider traps" or "black holes" -- sites where each page has links to many more programmatically-generated pages, without any useful content on them. The classic example is a calendar that keeps going forward through the 21st Century, although it has no events set after this year. This can cause the search engine to waste time and effort, or even crash your server.
• Readable URLs are good for more than being found by local and webwide search engine robots. Humans feel more comfortable with consistent and intuitive paths, recognizing the date or product name in the URL.
• By abstracting the public version of the URL, it will not be dependent on the backend software. If your site changes from Perl to Java or from CFM to .Net, the URLs will not change, so all links to your pages will remain live.

Source: http://www.searchtools.com/robots/goodurls.html

So how do I do it?
I will not be going into the details behind the URL Rewriter as Scott Mitchell’s article on MSDN explains it in great detail. But I highly recommend you read the article…it’s very good. This article is for those of you who want to implement it in 10 minutes and be done…and then maybe learn about it later.

Download Code
Download Binaries only

1. Update your web.config
In order for the Rewriter code to execute, you must add an HTTP module entry into the web.config. You can also use an HTTP handler (see MSDN article for explanation).



2. Setup your rules
For URL Rewriting to work, you need a “LookFor” value (the URL being sent over in the browser) and a “SendTo” value (the URL to rewrite to). In the web.config, it’s very easy. First, add a reference to your rules configuration section:



then add your rules:



In the database, it’s also very easy, but requires a little work. First, create a table:



Then, write a stored procedure to retrieve all Rewriter Rules:

CREATE PROCEDURE dbo.spRewriterRule_GetAll

AS

SELECT * FROM RewriterRules


Be sure to update the "RewriterUseDB" setting in the web.config based on your decision to use the web.config or database.



A few points about Rules:

• The “~” is a built-in feature that means “go to the root level of the application”.
• All entries must be XML compliant. So be sure to escape periods (\.) and excode ampersands (&amp;).
• Be sure to take the time to design your rules and folder hierarchy. On brianpautsch.com, I currently have the Rule “~/ShowItems(\d{4})/(\d{2})/(\d{2})\.aspx “to go “ShowItems.aspx? Year=$1&amp;Month=$2&amp;Day=$3”. I think I’m going to change it to “LookFor” “~/(\d{4})/(\d{2})/(\d{2})/” so people can type in bp.com/2006/03/01/ to get the entries they’re looking for. Both LookFor’s are intuitive, but the latter is much better.

Note: Rewriter rules that have folders in them must have the folder exist! And in that folder must be a default page with a simple “<%@ Page %>” tag.

2. Reference the RewriterDB and ActionlessForm DLLs
I recommend you add the projects to your solution and then add references. Then when you run the code, you can step through the Rewriter code.

3. Create your webpages
Based upon your rules, create the necessary webpages to support them. Be sure to add the ActionlessForm “form” tag at the top of each webpage:



And then add a few links to your webpage to test your rules out.



This allows the page to post back to itself without revealing the true path.

A few words on URL Rewriting…
I have implemented this into many websites and it works great. A couple websites have several thousand rewriter rules and there is no sign of performance loss.

• When possible, use regular expressions over exact matches.
• When using the database approach, be sure to associate each rewrite rule with the information it is rewriting. For example, if each blog entry has its own rewrite rule, you’ll need to add a column titled “RuleID” to the BlogEntries table. This is necessary so that you can JOIN on the RewriterRules table when displaying the blog entries and also easily find the rule when updating/deleting blog entries.
• When using the database approach and your “LookFor” has subfolders, be sure you create these! In the code provided, I simply created a folder titled “defaultpage” and whenever I create a rule, I ensure the folder exists and the file exists. Here’s some code I have written for live applications:

1//Ensure rewriter path exists2DateTime dteCreatedOn = objDataSvc.CreatedOn;
3Helpers.EnsureDefaultFileExists(Server.MapPath("\\"), 
4"Blogs\\" + String.Format("{0:yyyy}", dteCreatedOn) + "\\" + 
5 String.Format("{0:MM}", dteCreatedOn) + "\\" + 
6 String.Format("{0:dd}", dteCreatedOn) + "\\" + 
7 txtPageName.Text.Trim() + "\\Default.aspx"); 
8//Clear cached values so they reload on next page click from anyone9HttpContext.Current.Cache.Remove("RewriterConfig");

1publicstaticvoid EnsureDefaultFileExists(string strRoot, 
string strFilePath)
2{
3//Directory missing4if (!Directory.Exists(Path.GetDirectoryName(strRoot + strFilePath)))
5  Directory.CreateDirectory(Path.GetDirectoryName(strRoot + strFilePath));
67//File missing8if (!File.Exists(strRoot + strFilePath))
9 {
10string strDefaultLoc = 
11   ConfigurationSettings.AppSettings["RewriterDefaultPage"].ToString();
12  File.Copy(strRoot + strDefaultLoc, strRoot + strFilePath, true);
13 }
14}

Looking to automate your web testing? Create your own Windows forms robot to test your website (and continue to test as you add more and more features!). Here's a basic example to get you started today:

BE SURE TO CHECK OUT PART 2 OF THIS ARTICLE WHERE FRAMES ARE INVOLVED

Download code

1. Create a new Windows Forms application and add in the "Microsoft Internet Controls" COM Component (shdocvw.dll).

2. Add a "Search" textbox, a couple of radio buttons, a web browser and a "Go" button. On click of the "Go" button, add the following code:

1privatevoid cmdGo_Click(object sender, System.EventArgs e)
2{
3//Navigate to the page4 Object objNull = null;
5 WebBrowser.Navigate("http://www.google.com", ref objNull, ref objNull, ref objNull, ref objNull);
6 mbSearching = true;
7}

3. Handle the web browser's "DocumentComplete" event like s

1privatevoid WebBrowser_DocumentComplete(object sender, AxSHDocVw.DWebBrowserEvents2_DocumentCompleteEvent e)
2{
3//If we're not searching, exit4if (!mbSearching)
5return;
67 HTMLDocument myDoc = null;
8try9 {
10//Get the web browser document11  myDoc = new HTMLDocumentClass();
12  myDoc = (HTMLDocument) WebBrowser.Document;
1314//Find the textbox15  HTMLInputElement objTextBox = (HTMLInputElement) myDoc.all.item("q", 0);
16  objTextBox.value = txtSearchFor.Text;
1718//Click the selected button19string strBtn = (rdoClickSearch.Checked ? "btnG" : "btnI");
20  HTMLInputElement btnSearch = (HTMLInputElement) myDoc.all.item(strBtn, 0);
21  btnSearch.click();
22 }
23catch (Exception ex)
24 {
25  MessageBox.Show("Error:" + ex.Message);
26 }
27finally28 {
29//Release memory30  myDoc = null;
31 }
3233//Clear "searching" flag34 mbSearching = false;
35}

Some example screenshots

Yeah yeah yeah...I know...I need to get this done!

I finally published this article...
see: http://www.brianpautsch.com/ShowItem48.aspx

(I created a new entry so that RSS subscribers would see the new article.)

There are a lot of industries that need to show their data results based on proximity to a specified zip code. Some include car dealers, real estate agencies, job listers, etc. A few of my customers have asked for this capability and instead of buying a boxed solution and subscribing to a zip code list, I decided to build it myself, maintain it myself and bill each customer for usage. Actually, it's not that difficult. Once you figure out some of the radius and distance calculations and where to get current zip code data from, it's as simple as creating a few SQL Server User Defined Functions and a service to periodcially update your database.

Download code

Sample House Search Application


Database
At a minimum, you'll need a database with zip codes and their latitude and longitude coordinates. My example includes a database with all of the zip codes for Illinois along with the City, State, Latitude and Longitude.

Functions
RadiusFunc : Accepts @ZipCode (zip code) and @Miles (distance from zip code in miles) and returns the Maximum Latitudes and Longitudes for the radius of the zip code.
DistanceFunc : Accepts Latitudes and Longitudes for two zip codes and returns their distance Miles.

Stored Procedures
spHouses_GetNearZipcode: Accepts @ZipCode (zip code) and @Miles (distance from zip code in miles) and returns the houses listed in the radius.

Default.aspx
Default.aspx: Simple example where the user enters in a zip code, selects a distance from zip code radius and clicks enter to get results. Results are then bound to a repeater.

Default.aspx.cs
Default.aspx.cs : Nothing special about this code...the real logic for zip code searching is in the database!

A few months ago, a business partner and I sold our eBay drop-off store. We opened it in July of 2004 and it was the first eBay drop-off store in the Chicagoland area (maybe all of IL). It was working out OK, but we realized it would never be a huge money maker. The new owner just opened a new location in Manhattan...check it out: http://www.yourauctionpros.com

When we had the store, we had Yahoo DSL. I believe they wanted $40 or $50 for a static IP Address. It wasn't that important we have one, but we wanted one. I developed software for the store that managed the eBay auctions (it talked to eBay via APIs), automated the image resixing, created templates, talked with PayPal (also via APIs), managed out customers, balanced out acounting, etc. Well the software was an ever evolving product which required me to log into the system and migrate new versions. A dynamic IP Address would not do as migrations usually occured during non-business hours.

Solution: IP Address Sender - A windows service that checks the IP Address in the router and emails it to desigated email addresses when it changes.

Download code - Windows version to test with
Download code - Windows Service

Application configuration settings
App.config: The app.config file contains all of the settings you need to make this work for a Linksys router. I have tested it with two models (WRT54G - Wireless-G Broadband Router and BEFCMU10 - Cable Modem with USB and Ethernet Connections). The key names are pretty self-explanatory, but here are a few of the important ones:

Interval - How often will the service check (in milliseconds)

EmailUserName and EmailPassword - For email authentication, if necessary

RouterPage - Page in router intranet site that has the IP Address listed

RouterUserName - Username for logging into router intranet site. Note: Linksys default is " ", not ""

RouterPassword - I believe the Linksys default is "admin"

RouterPatternStart - Regular expression pattern before the IP Address to match

RouterPatternEnd - Regular expression pattern after the IP Address to match
The Start/End Patterns matches are needed since the router page may list multiple IP Addresses.

Main class that does everything
clsMain.vb: The code is well commented, but a few things are worth noting:

Line 115: WebRequest object is used to get the router's HTML

Line 117: Credentials are supplied since the router's intranet sites requires authentication

Line 124-128: A regular expression is used to find a match on the IP Address

Line 136: Saves the IP Address to a physical file at the install path

Line 142: If the IP Address has been updated, an email is sent

Line 166: The new IP Address is saved to the physical file

A word on Windows Services...
Also included in the download zip file are two batch files which make installing and uninstalling the Windows Service a breeze. Be sure to update the paths first! Also, when Windows Services are installed and started, the ApplicationName.exe.config settings are loaded into memory. Any changes to this file will not take effect unless the service is uninstalled and reinstalled.

While building websites over the past few years, many customers have asked for a seamless PayPal integration. Until a couple years ago, this wasn't very easy. Now that PayPal has their library of Web Controls and API interfaces, integrating an ecommerce website with PayPal is very easy. This example website shows you how to do it.

Getting started:
1. Download and read the PayPal API Integration Guide
2. Create a PayPal Developer account at http://www.paypal.com/cgi-bin/webscr?cmd=p/pdn/intro-outside
3. Create a PayPal account (if you don't have one already) and get your account Verified.
4. In your PayPal account, under 'Profile -> API Access', create an API certificate and convert it to a PDT key at http://www.paypaltech.com

Download code

1. Webpage interface and code behind
I created a simple DataGrid that binds to a DataTable. To allow for RadioButtons in a datagrid, I chose to use the MetaBuilders custom RadioButton control.


Lines 57-75: On Page_Load, not IsPostBack, I create a DataTable of IPod products, bind them to the DataGrid and Save them to Session State. Also, we assign the PayPal email address to the checkout button on Page_Load.
Lines 115-141: This event fires by the PayPal UploadCompleteCartButton button. To pass shopping cart items to PayPal, create and populate a "CartItem" object for each product and add them to the UploadCompleteCartButton button. Then, populate the "CustomerInfo" object in the UploadCompleteCartButton. And finally, populate the "Custom" property with the order number key from your system and set the return URL to "PDTHandler.aspx". PDTHandler.aspx is a page we will create to read PayPal's response to our transaction.

Upon clicking the "PayPal Checkout" button...
The user is taken to PayPal.com where they can login, create an account or simply checkout (PayPal no longer requires an account to make a credit card purchase!).


Upon logging in...
The user will see the product they selected and can checkout (Note: change code to send description only!).


2. Upon Checkout...PDTHandler.aspx
When the user completes the transaction (or cancels), PayPal will automatically reroute them to the "ReturnUrl" specified in the UploadCompleteCartButton's click event.
Lines 37-53: On Page_Load, we check to see if a transaction was returned (if not, the user canceled the PayPal transaction). If there is a transaction, we call "GetPDT" to get the transaction details.
Lines 57-88: GetPDT makes an HttpWebRequest out to PayPal and receives a stream of text back. This text string is delimited with a newline and carriage return. The first line returns "SUCCESS" or "FAIL" and the remaining lines are the details. Each detail line is a named value pair collection delimited with an equal sign. Details on the response can be found on page 101 of the PayPal API Integration Guide.
Lines 92-13: GetPDTValue is a method I included to help you find specific transaction return values. Just pass in the PDT return string and the key and the value will be returned.

I finally took this time to add more features to this application...and I think I'm done.
New features added:
 - Changed filtering to directional arrows (<, >, =)
 - Date span filtering
 - Datagrid color coding
    - White: not checked to migrate
    - Blue: checked to migrate
    - Green: current selected row
    - Red: files that might not want to be migrated (i.e. web.config)
 - Files that might not want to be migrated (i.e. web.config) are not checked to migrate by default

Download application

Below is a partial screenshot of the application (click image for large view).

 

Features coming soon:
Impersonation - Maybe one day...if I find time...

Please download and try the application out.

Everyday, I'm working on anywhere from 1 to 5 personal or customer websites. When it's time to migrate changes made that day or over a period of time, it's very important to migrate everything that has changed. In the past, I simply compared the modified dates and migrated anything newer to the detination server. But that's not foolproof. What if the servers are on a different time zone. What if someone accidentally changes the date or time on your computer. To prevent these migration problems, I created an application that compares source and destination folders and determines what needs to be migrated in what direction.

Download application

Below is a partial screenshot of the application (click image for large view). It allows you to save profiles (different source/destination combinations for development, staging and production environments, for example). It also allows you to display only production files (.htm, .aspx, etc., NOT: .cs, .ascx, etc.) and which direction files need to be migrated. It also warns you when the web.config needs to be migrated (so you migrate it manually).

 

Features coming soon:
Impersonation - Migrating files over a secure network doesn't work without the right credentials.
Date filtering - Ability to filter display by date range.

Please download and try the application out. It's a work in progess, so please let me know what you think.

One of my customers wanted to make their site searchable. They have a lot of content in different places (physical files, other websites, database, etc.). Trying to search all of these places real time would be a nightmare...and incredibly slow! So instead, I decided to build a web spider that caches the website content to a local drive on the web server and indexes the content into a Lucene index. The searches are now incredibly fast (under 1/2 sec), have relevancy scores and are merged. Some of the code was example code found at DotLucene (http://www.dotlucene.net/download/)...most of it is original.

This application shows you how to use Lucene to create a custom advanced search engine. There is way too much code to go over every part so I'm only discussing the important parts.

Download code 

Admin page - Allows user to index/re-index websites, delete indexed websites and index physical hard drive folders.

"http://www.brianpautsch.com" is indexed in less than 5 seconds.


"C:\_Websites\" is indexed in less than 5 seconds.


Lines 138-153: AddWebPage method: The spider calls this method for each link found. This method strips off any bookmarks, verifies the file extension is in our list of valid extensions and ensures the site is within our base URL. If all of these tests pass, an instance of WebPageState is created (URL loaded in constructor), a unique ID is assigned and the page is put in the queue of pages that need to be visited and indexed/cached.
Lines 171-214 : Process method: This method makes a WebRequest to the URL, checks the status code, stores the HTML and sets the process success flags.
Lines 261-269 : HandleLinks method: This method uses a regular expression to find all URL links on the page.
Lines 272-285: AddWebPageToIndex method: Once all of the information for the page is gathered this method is called to add the information to the index. Note that some fields are added as "UnIndexed", "Text", etc. Here's a little explanation on each:
    Field.Keyword - The data is stored and indexed but not tokenized - (Last modified date, filename)
    Field.Text - The data is stored, indexed, and tokenized - Searchable small text (Title)
    Field.UnStored - The data is not stored but it is indexed and tokenized. (File content)
    Field.UnIndexed - The data is stored but not indexed or tokenized. (URL)

Index/Cache Storage
As new websites are indexed, they are stored in separate folders under the "...\LuceneWeb\WebUI\cache\" and "...\LuceneWeb\WebUI\index\" folders. The current version only allows for one index of the hard drive and it's stored in the "...\LuceneWeb\WebUI\index\folders\" folder. This logic could be easily changed to have multiple indices of the hard drive.
 

Search page - Allows user to select an index and search it.

"http://www.gotdotnet.com" is searched for "microsoft" - 158 results found in .26 seconds.


"C:\_Websites\" is searched for "search" - 10 results found in .63 seconds.
 

Lines 145-217 : PerformSearch method - This is the main method for this class (code-behind). It starts off by determining the index location and creating an empty datatable of results. A basic query is performed (no special filtering, i.e. datetime, buckets) and returns a "Hits" object. A QueryHighlighter object is created and as each result is extracted the contents are highlighted. Finally, the datarow is saved to the datatable and later bound to the repeater.

After writing dozens of data tiers for many websites and applications, I felt it was time to write a data tier generator for myself. About 5 years ago, I created a data tier generator for VB6, but never really found time to create one for C#...until now. This generator has already saved me countless hours from writing classes (properties and methods) and stored procedures. Not only are they done for me now, but they're 100% accurate!

Download application

1. Run data tier generator
The application asks for three things:
Connection String: database you wish to generate the data tier for.
Output Path: location to generate the code to.
Project Name: name of the project. Application will create folder, if not found.



2. Add MDAB to project
Once the application completes, the output path will have a folder for the project you just created. Open the project and simply add in the Microsoft Data Access Blocks. The project will compile and is ready to be accessed from a UI or Business layer.



3. Create the Stored Procedures
Finally, execute the SQL scripts in the "Data" folder to create the stored procedures.



What is generated?
The data tier should encapsulate all database information and expose all data through objects (classes) or properties. This is exactly what my data tier generator does. All database fields become public properties and all data is received/returned in objects. Extended the classes is very easy once the initial generation is complete...and if you decide to change a table drastically, simply rerun the generator.

A recent project at my current contract required us to research and obtain a 3rd party advanced search engine to fulfill some of the project requirements. I was not involved in the research, but the company decided to go with Verity. It turned out the Verity K2 Enterprise software had everything we needed to get the project developed and deployed fast. Working with the Verity software was very interesting and I decided to see what else was out there (open source only). Almost immediately, I came across the Apache Lucene project. "Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform.". Getting the Java examples to work was very easy and so I started to look for more online...that's when I came across the .NET version at SearchBlackBox.com. "SearchBlackBox Lucene Edition is a 100% C# based native .NET assembly that is fully optimized for the .NET Framework.". The website doesn't offer a lot of example, but it's easy to get it working. Also, the software product Lookout (aquired by Microsoft in June 2004) runs on the Lucene.NET code.

This application walks you through a simple implementation of the Lucene .NET DLL. We're bascially going to build out own Desktop Search Engine.

Download code

1. LuceneEngine.IndexFiles.cs
Lines 12-59: Input public properties (FilesLocation, IndexLocation, StopProcessing) and output public properties (Error, NumDocsIndexed, NumDocsSkipped, NumDocsErrored, TotalTime).
Lines 63-68: Custom events that are fired when the percentage of files completed changes and when errors occur.
Lines 69-135: StartIndexing is the entry method to get the indexing process going. After validating the input properties, I iterate through the directory chosen to index and all of its subdirectories. After I have all of the directories stored in mstrSubDirectories, I iterate through each one and call IndexDocs.
Lines 136-182: IndexDocs is the real worker in this application. This method loops through each file and, if the extension is supported, adds the document to the index.

2. LuceneEngine.MakeFileDoc.cs
Lines 12-46: The only method here is Document(), which returns a Lucene.Net.Documents.Document object. A Lucene Document contains multiple properties and the code populates some of them: filename, path, name, length, contents, creation_time, last_write_time and last_access_time.

3. LuceneEngine.SearchFiles.cs
Lines 16-72: Input public properties (IndexLocation, SearchFor, LastWriteFrom, LastWriteTo, NumHitsRequested) and output public properties (NumHitsFound, ResultsXML, ResultsDataView, TotalTime, Error).
Lines 73-158: The only method here is StartSearch(). This method creates an instance of the IndexSearcher and StandardAnalyzer (other Analyzers are available, but the StandardAnalyzer is basic enough for this project). It then creates a Lucene.Net.Search.Query object and selects the "contents" to be searched. Then the filtering criteria is loaded into a Lucene.Net.Search.DateFilter object. Finally, we call the IndexSearcher's Search method and return a Lucene.Net.Search.Hits object. From that, I get the number of hits and can iterate through the results collection. In this example, I create an XML resultset and a DataView (gives more flexibility to the consumer, i.e. WinForms App, Website or Web Service). The success result (true/false) is returned.

4. LuceneUI.frmMain
The top section consists of the indexing criteria. The "Files Location" is the root folder for the files you want to index. The "Index Location" is the location where the index should be stored.

 

The middle section consists of the search criteria and results. Simply enter in the search phrase and date range and the results will return immediately. To launch any item, double-click the row.

 

5. LuceneUI.frmMain.cs
Lines 383-478: The cmdStart_Click event handles all of the processing to index the files. After validating the input data, it creates an instance of LuceneEngine.Index.IndexFiles, sets up the events, loads the properties and calls the StartIndexing method. Upon return, it displays the results. While indexing was occurring, events were being fired constantly and the IndexFiles_OnPercentCompleteChangedHandler (Lines 486-497) method was processing them.
Lines 506-618: The cmdSearch_Click event handles all processing to search  the index. After validating the input data, it creates an instance of LuceneEngine.Search.SearchFiles, load sthe properties and calls the StartSearch method. Upon return, it displays the statistical results in a label, binds the results DataView to the DataGrid and automatically resizes the columns so they can be viewed (SizeColumnsToContent).

Using custom objects is always a great practice, especially in n-tiered development where abstraction across the layers is crucial. All too often I see people (including myself) use custom objects to handle most of the logic, but then databind repeaters, datagrids, etc. with datasets, datatables and data readers. By doing so, the database and user interface are now tightly coupled and maintenance can become a headache later on.

Download code

What's the solution? Bind your UI controls with your custom objects. Here's how:

1. UserColl.cs - Create a class that defines your custom object (UserDetails)
Lines 6-34: A simple class with three members (Email Address, First name and Last Name)

2. UserColl.cs - Create a custom collection class that implements the IEnumerable interface (Users)
Line 37: Inherit System.Collections.IEnumerable interface
Line 40: Declare private ArrayList to store data
Lines 43-46: Return the enumerator from my ArrayList
Lines 49-109: Methods to add/update/retrieve objects from the collection

3. WebForm1.aspx - Implement into a WebForm


Lines 8-11: I added JavaScript that confirms the delete.
Lines 23-43: DataGrid TemplateColumn's - Notice that each item is pulled from the Container and is referenced by the member name.
Line 47: TemplateColumn for the Delete LinkButton - Coded this way so was can find it during the ItemBound event and add the JavaScript confirm function.

4. WebForm1.aspx.cs - Implement into the CodeBehind
Lines 47-52: On first load, call LoadNewUsers to, you guessed it, load new users into the object. In a real life scenario, this would be pulled from a database.
Line 55: For this example, I simply put the session's data into a Session object.
Lines 58-86: LoadNewUsers - Create 3 objects and load them with details...each being added to our collection class. At the end, store this in Session State (for this example).
Lines 94-99: Look familiar? Usually, you bind with a dataset, datatable, etc...this time you're binding with a custom object!
Lines 101-114: dgUsers_ItemCreated - just a little something extra. When DataBind is called, the ItemCreated event is caught here. For each item, I find the Delete LinkButton and and the JavaScript function as an "onclick" attribute.
Line 116-167: dgUsers_ItemCommand - Some examples on how to access the collection. When "View" is clicked, I retrieve the object from the collection by "index" (Line 124). When "Delete" is clicked, I retrieve the object from the collection by "Email Address". To do this, I first need to find the LinkButton (Line 140). Then I can retrieve the object (Line 144). Once I have the object, I can delete it (Line 146). Finally, I rebind the custom object to the DataGrid to reflect the changes (Line 163) and refresh the Session variable (Line 166). For this example, the rebind and session variable refresh should just be in the "Delete" if structure, but I had originally planned to have an "Edit" link.