MSDN Blogs: CLRs, Web Services and JSON – Design Decisions

Recently I was working on a CLR that needed to talk to a web service and return a tabular dataset. Within the development and deployment process there were several considerations concerning security and design. This post will documented these to help understand the challenges and options available.

Security problems should be considered early as they will colour how the CLR is developed.

If the CLR uses a DLL that is not installed on the SQL Server then the DLL will need to be installed. (A list of supported libraries in SQL can be found here https://msdn.microsoft.com/en-us/library/ms403279.aspx)

For an unsafe assembly the database has to be set to Trustworthy, this is a non-default security setting so some companies are reluctant to change.

If the database cannot be made Trustworthy then the CLRs may be deployed to MSDB as MSDB is Trustworthy by default. The extended function or procedure can be referenced from the user database with the syntax msdb.dbo.<CLRProcName>. However, bear in mind if using Always On the MSDB database will not move across in a failover event. To achieve high availability you will need to deploy to all replicas that might be failed over to.

Using third party DLLs can sometimes be worked around by refactoring the code.

This example CLR needs to call the free movie database webservice http://www.omdbapi.com/. When called this websevice returns JSON in the following format

http://www.omdbapi.com/?t=Simply+Irresistible&y=&plot=short&r=json

{"Title":"Simply Irresistible","Year":"1999","Rated":"PG-13","Released":"05 Feb 1999","Runtime":"96 min","Genre":"Comedy, Drama, Fantasy","Director":"Mark Tarlov","Writer":"Judith Roberts","Actors":"Sarah Michelle Gellar, Sean Patrick Flanery, Patricia Clarkson, Dylan Baker","Plot":"A magical crab works wonders for a terrible chef's culinary skills, leading her towards the man of her dreams.","Language":"French, English","Country":"Germany, USA","Awards":"N/A","Poster":"http://ia.media-imdb.com/images/M/MV5BMTYyNTg3Mzg2M15BMl5BanBnXkFtZTcwNzczNjUyMQ@@._V1_SX300.jpg","Metascore":"27","imdbRating":"5.3","imdbVotes":"11,290","imdbID":"tt0145893","Type":"movie","Response":"True"}

Usually when dealing with JSON it is common to use Json.NET from Newtonsoft. Json.Net provides quick and easy methods to convert JSON into an object that can then easily be converted to a tabular format for SQL. However, this could not be used in this case due to the security policy as it would require the deployment of a DLL on the SQL Server.

SQL 2016 has new functions for handling JSON. The CLR would only need to return the JSON string and then OPENJSON() could be used to convert the data. As shown in the following diagram:

However, in this case we were using a previous version of SQL.

Without SQL 2016 and without deploying Json.Net to the server the only option was to code in a way as to convert the JSON either within the CLR or within SQL Server with TSQL. Returning a tabled valued function from a CLR was a cleaner solution and as the JSON format returning from the web service was simple. This was achieved with simple string manipulation. (A regular expression could have been used but looking at the JSON we could break up into two columns using a few simple string replacements and Splits)

The resulting solution is as follows:

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.Collections;

public partial class StoredProcedures
{
    [SqlFunction(FillRowMethodName = "OMDSearch", TableDefinition = "Name nvarchar(400), Value nvarchar(4000)")]
    public static IEnumerable tvfnOMDSearch(String title, String year)
    {
        System.Net.WebRequest req = System.Net.WebRequest.Create(String.Concat(@"http://www.omdbapi.com/?t=", title, @"&y=", year, @"&plot=short&r=json"));

        System.Net.WebResponse resp = req.GetResponse();
        System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());

        string jres = sr.ReadToEnd().Trim();
        /*Remove JSON brackets and mark the columns with a not(¬) sign*/
        jres = jres.Replace("{"", "").Replace(""}", "").Replace("":"", "¬");
        /*Split out the lines into an array*/
        string[] movieinfo = jres.Split(new string[] { "","" }, StringSplitOptions.RemoveEmptyEntries);

        return movieinfo;
    }
    public static void OMDSearch(Object obj, out SqlString Name, out SqlString Value)
    {
        string[] resarr = obj.ToString().Split('¬');
        Name = resarr[0].ToString();
        Value = resarr[1].ToString();
    }
}

This may be called as follows:

CREATE TABLE #tbMovies (MovieYear varchar(150), MovieName varchar(150))
GO

INSERT INTO #tbMovies VALUES
('2014',	'God''s Pocket'),
('1972',	'Malcolm X'),
('2012',	'Alter Egos'),
('2008',	'Step Up 2: The Streets'),
('2015',	'LEGO Friends'),
('2012',	'Nate & Margaret'),
('2014',	'Castle'),
('2012',	'Jack and Jill'),
('2016',	'Carol'),
('2012',	'Dragons'),
('2011',	'Beastly'),
('2015',	'April 9th'),
('2015',	'A Girl Walks Home Alone At Night'), 
('2012',	'People Like Us')
GO
SELECT * FROM #tbMovies AS T
CROSS APPLY MSDB.dbo.tvfnOMDSearch(T.MovieName, T.MovieYear)

To give the following result: