Logging within Azure via Linkedin

Recently there was a discussion on Linkedin about logging within Azure.  The discussion brought up the use of Enterprise Library and I wanted to share the response I provided.

Posted on Linked:

The things you are talking about are many of the driving forces behind Unified Logging (http://www.unifiedlogging.com/) and let me explain why.

Yes adding an email notification with Enterprise Library is easy though if thousands of email “events” (could be errors) happen you get thousands of emails.  On top of that to change when an email is sent you have to go touch production to make the change.  Lastly, you may not be the only one who wants to be notified for specific events, say application support wants to be notified when a specific event happens every evening but you do not really care about that.

I’m a firm believer of disconnecting the notification configuration and notifications themselves from the application itself and let people get notified when they want to be notified.

I too use Enterprise Library 5 and since Unified Logging is not a logging framework but rather a bucket for your logging information it comes down to about 10 seconds or work to add the Unified Logging NuGet Package and the one line of configuration in the logging configuration for entlib (http://blog.unifiedlogging.com/getting-started-with-net-nuget-package/).  Don’t forget to add the listener to category sources in the entlib config.   Next setup a notification index (http://blog.unifiedlogging.com/indexes-102-notifications/ ), most likely adding a notification to the “error” index that is created for you will get you started.  Most .NET applications (though there are many connectors for many languages) will just use the trace listener that ships with the NuGet package and leverage the logging framework of their choice.

I would point out that you are not limited to one trace listener as it is a collection. I myself use the event log listener, Azure Diagnostics Listener, Unified Logging Listener, and a couple custom listeners.

It is also worth pointing out that order of listeners for a category source does sometime matter with entlib, if the Event Log listener throws (and it could) the other listeners for the category source will not process so it is best to put that listener last in the category sources.

Comments Off

Resiliency with Azure Caching

Using Azure caching can be easy when everything works though there have been many times when everything has not worked.  This post is applicable to dedicated caching (previously known as caching preview) and shared caching.  So how do you build in resiliency to your caching?

First, let’s assume data is being cached from a SQL Azure database, though what is outlined in this post is applicable to other data stores as well. First off let’s outline what could go wrong.

1) The cache becomes unavailable (could be because of a variety of reasons)

2) The database become unavailable

There are many reasons for number one to happen and the easiest way to handle it is to fall back to retrieving from the database.  You can have code that looks something like the below:

public RealyCoolCacheItem GetRealyCoolCacheItem(string key)
{
     try
     {
          var cacheItem = GetCacheItem(key);

          if (cacheItem == null)
          {
               var reallyCoolItem = RetrieveReallyCoolCacheItemFromDb(key);

          if (reallyCoolItem != null)
          {
               DataCache.Put(key, reallyCoolItem, TimeSpan.FromHours(2));
          }

               return reallyCoolItem;
          }
          else
          {
               return (RealyCoolCacheItem)cacheItem.Value;
          }
     }
     catch (Exception ex)
     {
          //Log the Error
     }

     //Would end up here if working with the cache throws
     return RetrieveReallyCoolCacheItemFromDb(key);
}

The second situation of a database going down is a bit more interesting.  The common direction is to have a fail over database though this is a big stick for a simple problem.  The cache is used to read frequently used data.

The solution Unified Logging has implemented is to serialized cache data to blob storage each time it is retrieved from the database overwriting the existing blob if necessary.  If the retrieval from the database fails it then reads the last blob written and things keep working.  This is how the submission endpoints of Unified Logging keep up and running when these undesirable events occur.

private RealyCoolCacheItem RetrieveReallyCoolCacheItemFromDb()
{
     try
     {
          //Retrieve from DB HERE
          var reallyCoolDbItem = magicDb.GetReallyCoolItem();

          //Save the item to the secondary failover datastore
          //In this case the secondary store is blob storage
          // RetrieveReallyCoolCacheItemFromDb_Failover is a constant
          failover.Save(RetrieveReallyCoolCacheItemFromDb_Failover, reallyCoolDbItem);

          return reallyCoolDbItem;
     }
     catch (Exception ex)
     {
           //Log the error

          //The retrieval from the db has failed so get the item form the secondary datastore
          return failover.Retrieve<RealyCoolCacheItem>(RetrieveReallyCoolCacheItemFromDb_Failover);
     }
}

At this point you are probably saying, all well and good but what is “failover”.  It is a simple class which implements a simple interface which has Save and Retrieve.

public interface ICacheFailover
{
      void Save<T>(string key, T serializableObject) where T : class;

      T Retrieve<T>(string key) where T : class;
}

Comments Off

Creating a Custom Connector

Creating your own connector is simple.  First step is to download the source for a current connector as a guide.  This blog post will go into this topic step by step though many times it is helpful to have the full source of an implemented connector to look at.

For the purpose of this post the code samples will be from the php connector.

1)  If you desire a certain format for your messages make sure you do it prior to sending the message

2)  To start building the request you will need

    • create a hash of the content using your secret key
    • headers for your access key
      • hash
      • content length
      • content type being application/json.

3)  Create the hash using your Secret Key (dots are for concatenation).  Note the false parameter on the sha1 function which will return a 40-character hexadecimal number.  Take note that the input format to create the hash is <message>|<secret key>

$validation_string = $message.”|”.UL_SECRETKEY;

$computed_hash = sha1( $validation_string , false );

4)  Json encode the message so it has the format {“Content”:”[your message]”}

$fields = array( ‘Content’ => $message );

$json_data = json_encode( $fields );

5)  Set the headers

curl_setopt( $ch, CURLOPT_HTTPHEADER,

array(‘Content-Type: application/json’,

‘Content-Length: ‘.strlen( $json_data ),

 ‘Authorization: ‘.UL_ACCESSKEY,

 ‘MD5: ‘.$computed_hash ) );

6)  Set the post data

curl_setopt( $ch, CURLOPT_POSTFIELDS, $json_data );

7)  Send the http request to your submission url

8)  Handle the response, a 201 means the http post was successful.

9) Feel free to reach out to support for help
Some things to consider

  • Send asynchronously where possible
  • If a 201 response is not received consider a retry
  • Log locally as well as to Unified Logging

For a an example of a more robust connector look at the .NET Trace Listener implementation which sends everything async and has a retry queue if a message fails for some reason.

Comments Off

New Feature: Create Share Links

create-link
There are times when you are looking at messages that came in and you would like someone else to take a look at it without sending them the whole message.  Now you can simply right click on the message and select Create Link.  A window will pop open with an encrypted link you can use for sharing and the viewer will be brought into Unified Logging in view only mode.

Enjoy!

Comments Off

New Feature – Auto Open Message When Link Is Followed

A simple yet beneficial update, now when you receive an email notification and follow the link to view on the web the message is automatically opened.

AutoExpand

 

 

 

Comments Off

NEW Feature – View Only Mode for Notification Links

view-onlyWhen you set up a notification on an index and then receive a notification it has a link to view it in the web portal.

Now, when you click on that link you will be taken in as a view only user and can only see messages, trends and regions.  You will know you are in view only mode if you see the green bar.

Why is this important?  The notification will likely be shared, either the email is forwarded or it is copied / pasted into a ticket.

Comments Off

Getting Started with Unified Logging WordPress Plugin

The WordPress plugin makes sending logging information from WordPress simple.  Ever wonder why your WordPress site is slow?, it could be a plugin you installed but without looking at the logging output you would not know.

  • If you do not have an account go sign up and get collecting
  • Start by downloading the WordPress plugin from WordPress.org
  • Install the plugin via the plugins page in your wordpress site
  • From your profile page get the Submission Url, Access Key and Secret Key
  • In the Unified Logging settings page enter your  Submission UrlAccess Key and Secret Key
  • Review the logging level
    • The default level is set to exclude E_NOTICE and E_DEPRECATED, we recommend this not be changed unless you are excluding more logging levels
    • If using earlier than PHP 5.3  E_NOTICE and E_STRICT will be excluded
    • Including more logging levels can cause your site to run slow because it is logging a lot
    • Many plugins and themes log a lot of E_NOTICE and E_DEPRECATED messages which is why they are excluded
    • More info on the levels can be found at: http://php.net/manual/en/errorfunc.constants.php

RECOMMENDED INDEXES

  • Add a notification to the default Error index which a condition to notify once if matched within 5 minutes
  • Add a new notification index and type in the word “warning” with a condition to notify once if matched within 5 minutes

To Learn More About Indexes Visits These Posts

**This plugin will soon be on WordPress.org so you will be able to search and install from the plugin section of your site.

Comments Off

Getting Started with PHP Connector

The php connector is a simple stand-alone php file which you can “include_once(‘ul-error-handler.php’);” into a base php file.

  • If you do not have an account go sign up and get collecting
  • Start by downloading the php file from the use it page after logging in.
  • Unzip it and open it
    • From your profile page get the Submission Url, Access Key and Secret Key
    • In the ul-error-handler.php file update the UL_SUBMISSIONURL, UL_ACCESSKEY, UL_SECRETKEY with the values from your profile
  • Put this file in your website root directory
  • Add include_once(‘ul-error-handler.php’); to a php file (usually a config file for your site)
    • The file this is added to should be loaded for every page in your site
    • e.g. in wordpress the file to add this to would be the wp-config.php
  • Review the logging level set in UL_LEVEL
    • The default level is E_ALL & ~E_NOTICE & ~E_DEPRECATED, we recommend this not be changed unless you are excluding more logging levels
    • If using earlier than PHP 5.3 use E_ALL & ~E_NOTICE & ~E_STRICT
    • Including more logging levels can cause your site to run slow because it is logging a lot
    • Many plugins and themes log a lot of E_NOTICE and E_DEPRECATED messages which is why they are excluded
    • More info on the levels can be found at: http://php.net/manual/en/errorfunc.constants.php

RECOMMENDED INDEXES

  • Add a notification to the default Error index which a condition to notify once if matched within 5 minutes
  • Add a new notification index and type in the word “warning” with a condition to notify once if matched within 5 minutes

To Learn More About Indexes Visits These Posts

If you are rolling your own php website and you would like to send data to Unified Logging use the function ul_log_message and pass the message to log.

If you are using WordPress you have another option, check out the WordPress plugin.

Comments Off

Ditch the Monthly Subscription

Why pay for something you are not using or not using enough of!

Monthly subscriptions are a thing of the past for enabler products like Unified Logging.  By enabler products I mean software products that enable you to build better software products.

Unified Logging wants you to build better software products.  This is the motivation and passion behind Unified Logging and why we have a per message pricing model which is very affordable so everyone can take advantage of it.

 

Unified Logging – Logging for All

 

Comments Off

Unified Logging Offers a New North America Option!

Unified Logging is proud to announce the availability of a new data collection option for applications that are sending information within the North American Region.  In your profile you will notice the addition of a North America Submission Url which is optimized for the North American Region.

Enjoy!

Comments Off