Blog

I have created a helper class that will allow me to consume any XML or JSON request for deserialization into a class object. As you can see from the code below, the GetJsonRequest() and GetXmlRequest() methods allow you to pass an unknown type as well as the URL to where you are getting your request from. This makes things very straight-forward when you want to easily strongly type the data.

public class ApiWebRequestHelper
{
    /// <summary>
    /// Gets a request from an external JSON formatted API and returns a deserialized object of data.
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="requestUrl"></param>
    /// <returns></returns>
    public static T GetJsonRequest<T>(string requestUrl)
    {
        try
        {
            WebRequest apiRequest = WebRequest.Create(requestUrl);
            HttpWebResponse apiResponse = (HttpWebResponse)apiRequest.GetResponse();

            if (apiResponse.StatusCode == HttpStatusCode.OK)
            {
                string jsonOutput;
                using (StreamReader sr = new StreamReader(apiResponse.GetResponseStream()))
                    jsonOutput = sr.ReadToEnd();
                    
                var jsResult = JsonConvert.DeserializeObject<T>(jsonOutput);

                if (jsResult != null)
                    return jsResult;
                else
                    return default(T);
            }
            else
            {
                return default(T);
            }
        }
        catch (Exception ex)
        {
            // Log error here.

            return default(T);
        }
    }

    /// <summary>
    /// Gets a request from an external XML formatted API and returns a deserialized object of data.
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="requestUrl"></param>
    /// <returns></returns>
    public static T GetXmlRequest<T>(string requestUrl)
    {
        try
        {
            WebRequest apiRequest = WebRequest.Create(requestUrl);
            HttpWebResponse apiResponse = (HttpWebResponse)apiRequest.GetResponse();

            if (apiResponse.StatusCode == HttpStatusCode.OK)
            {
                string xmlOutput;
                using (StreamReader sr = new StreamReader(apiResponse.GetResponseStream()))
                    xmlOutput = sr.ReadToEnd();

                XmlSerializer xmlSerialize = new XmlSerializer(typeof(T));

                var xmlResult = (T)xmlSerialize.Deserialize(new StringReader(xmlOutput));

                if (xmlResult != null)
                    return xmlResult;
                else
                    return default(T);
            }
            else
            {
                return default(T);
            }
        }
        catch (Exception ex)
        {
            // Log error here.
            return default(T);
        }
    }
}

The ApiWebRequestHelper class relies on the following namespaces:

• Newtonsoft Json
• System.Xml.Serialization
• ​​System.IO;

The ApiWebRequestHelper can be used in the following way:

// Get Json Request
ApiWebRequestHelper.GetJsonRequest<MyCustomJsonClass>("http://www.surinderbhomra.com/api/result.json");

// Get XML Request
ApiWebRequestHelper.GetXmlRequest<MyCustomXMLClass>("http://www.surinderbhomra.com/api/result.xml");

Today, I stumbled across a really neat feature in Visual Studio 2015 that gives you the ability to create a strongly-typed C# class directly into your class library. I'm amazed that I've happened to overlook this key feature since Visual Studio 2012!

Better late than never.

In the past, when consuming large JSON data-structures, I normally head off to http://json2csharp.com to help me get the initial class objects generated, which works a treat. I've blogged about my experiences using json2csharp here.

The strongly-type class generator feature is hidden away in a place I would never have thought to look - Edit > Paste Special, where you will be given the option to either generate a class object based on XML or JSON. But it's in there for a reason.

All that needs to be done now is to either copy some XML or JSON ready to paste into a newly created class. I do find the generated output quite untidy, but this is a great starting point to generating complex data structures.

If you set a Controller's response type to "text/xml", you may encounter an: "XML Parsing Error: XML or text declaration not at start of entity". Your View may look something like this:

@{
    Layout = null;
}

<?xml version="1.0" encoding="utf-8" ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    @if (Model.Any())
    {
        foreach (SitemapNode node in Model)
        {
            <url>
                <loc>@node.Location</loc>
                <lastmod>@node.LastModified</lastmod>
                <changefreq>monthly</changefreq>
            </url>
        }
    }
</urlset>

In this case, I was creating a sitemap for one of my websites. So I created a Controller and View as I normally would do. However, when generating an XML output, you'll have to do something a little different in MVC:

@{
    Layout = null;
}<?xml version="1.0" encoding="utf-8" ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    @if (Model.Any())
    {
        foreach (SitemapNode node in Model)
        {
            <url>
                <loc>@node.Location</loc>
                <lastmod>@node.LastModified</lastmod>
                <changefreq>monthly</changefreq>
            </url>
        }
    }
</urlset>

Can you see what is the difference? You'd be forgiven for not seeing it. But if you look a little closer, you'll see that I pushed up my XML declaration right up next to where I set the Layout block. This is because Razor outputs extra lines within its markup.

So when I left an empty line after my Layout block (as seen my my first code example), this gets rendered as an empty line when you run the page which would not be valid XML.

Update - 28/08/2014

Just found an even better way to get around the same issue from reading Joe Raczkowski blog. All that needs to be done is place the main XML declaration at the top of the page inside the @{} braces:

@{
Layout = null;
Response.ContentType = "text/xml";
Response.Write("<?xml version=\"1.0\" encoding=\"UTF-8\" ?>");
}

I had around 2000 webpage URL’s listed in a text file that needed to be generated into a simple Google sitemap.

I decided to create a quick Google Sitemap generator console application fit for purpose. The program iterates through each line of a text file and parses it to a XmlTextWriter to create the required XML format.

Feel free to copy and make modifications to the code below.

Code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Xml;

namespace GoogleSitemapGenerator
{
    class Program
    {
        static void Main(string[] args)
        {
            string textFileLocation = String.Empty;

            if (args != null && args.Length > 0)
            {
                textFileLocation = args[0];
            }

            if (!String.IsNullOrEmpty(textFileLocation))
            {
                string fullSitemapPath = String.Format("{0}sitemap.xml", GetCurrentFileDirectory(textFileLocation));

                //Read text file
                StreamReader sr = File.OpenText(textFileLocation);

                using (XmlTextWriter xmlWriter = new XmlTextWriter(fullSitemapPath, Encoding.UTF8))
                {
                    xmlWriter.WriteStartDocument();
                    xmlWriter.WriteStartElement("urlset");
                    xmlWriter.WriteAttributeString("xmlns", "http://www.sitemaps.org/schemas/sitemap/0.9");

                    while (!sr.EndOfStream)
                    {
                        string currentLine = sr.ReadLine();

                        if (!String.IsNullOrEmpty(currentLine))
                        {
                            xmlWriter.WriteStartElement("url");
                            xmlWriter.WriteElementString("loc", currentLine);
                            xmlWriter.WriteElementString("lastmod", DateTime.Now.ToString("yyyy-MM-dd"));
                            //xmlWriter.WriteElementString("changefreq", "weekly");
                            //xmlWriter.WriteElementString("priority", "1.0");

                            xmlWriter.WriteEndElement();
                        }
                    }

                    xmlWriter.WriteEndElement();
                    xmlWriter.WriteEndDocument();
                    xmlWriter.Flush();

                    if (File.Exists(fullSitemapPath))
                        Console.Write("Sitemap successfully created at: {0}", fullSitemapPath);
                    else
                        Console.Write("Sitemap has not been generated. Please check your text file for any problems.");

                }
            }
            else
            {
                Console.Write("Please enter the full path to where the text file is situated.");
            }
        }

        static string GetCurrentFileDirectory(string path)
        {
            string[] pathArr = path.Split('\\');

            string newPath = String.Empty;

            for (int i = 0; i < pathArr.Length - 1; i++)
            {
                newPath += pathArr[i] + "\\";
            }

            return newPath;
        }
    }
}

I will be uploading a the console application project including the executable shortly.

From one of the projects I have been working on, I came across a snippet of code that used the XmlDocument.Load method. What alarmed me about this piece of code was the fact that there was no error handling. If for some reason the XML file could not be found or a node was missing, the whole page would have crashed. Not good.

I must admit, I am not exactly the best person to speak about implementing wide-scale error handling in every facet of code. But I do ensure the core foundations of an application or website do incorporate sufficient error handling.

So back to the matter in hand. This is the original code using the XmlDocument.Load functionality:

XmlDocument doc = new XmlDocument();

doc.Load(Server.MapPath("/xml/storeGB.xml"));

XmlNode countryNode = doc.SelectSingleNode("//countries");
foreach (XmlNode node in countryNode.ChildNodes)
{
    //Do something with the elements
    Response.Write(node.Name + node.InnerText);
}

I changed the code to the following:

XmlDocument doc = new XmlDocument();

//Check if language XML file exists
if (File.Exists(Server.MapPath("/xml/storeGB.xml")))
{
    try
    {
        doc.Load(Server.MapPath("/xml/storeGB.xml"));

        XmlNode countryNode = doc.SelectSingleNode("//countries");

        if (countryNode != null)
        {
            foreach (XmlNode node in countryNode.ChildNodes)
            {
                //Do something with the elements
                Response.Write(node.Name + node.InnerText);
            }
        }
        else
        {
            //Output error message if there is no node
        }
    }
    catch (XmlException ex)
    {
        Debug.WriteLine(String.Format("XmlException for countries: {0}", ex.Message));
    }
}

I am sure you will agree that this is the better approach to using XmlDocument.Load.

In 2005, the search engine Google launched the Sitemap 0.84 Protocol, which would be using the XML format. A sitemap is a way of organizing a website, identifying the URLs and the data under each section. Previously, the sitemaps were primarily geared for the users of the website. However, Google's XML format was designed for the search engines, allowing them to find the data faster and more efficiently.

Even the most simple sitemap to a website is quite important in order to allow search engines such as Google and Microsoft Live Search to crawl your website for any changes. The following example shows what a basic XML sitemap contains:

<?xml version="1.0" encoding="UTF-8" ?> 
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<url>
<loc>/blog/</loc> 
<priority>0.5</priority> 
<changefreq>weekly</changefreq> 
</url>
</urlset>

As you can see the sitemap contain the following:

• <loc> = Location of the page
• <priority> = The priority of a particular URL relative to other pages on the same site. The value for this tag is a number between 0.0 and 1.0, where 0.0 identifies the lowest priority page(s) on your site and 1.0 identifies the highest priority page(s) on your site.

The default priority of a page is 0.5.

• <changefreq> = This value indicates how frequently the content at a particular URL is likely to change.

Thankfully, there is a site that will automatically generate an XML sitemap for you: http://www.sitemapspal.com/

I have written a blog post a little while back on how to manually submit your sitemap to search engines which proves to be quite useful if you find that your site has not been crawled for a long time. You can find that blog post here.