Troubleshooting the insidious InvalidInput error

I've been plagued since the day I started working on Azure with occasional errors of type "InvalidInput" (such as this one). The detail of this errors says "One of the request inputs is not valid" and gives no other information. Obviously this means one of the properties on the entity is invalid for Azure Table Storage, but it does not tell you which one or why. A quick google will tell you lots of common mistakes that can cause this, but if those don't help you're out of luck.

I have come up with a function that will at the very least tell you which property is causing the error. It should be a method of a TableServiceContext subclass, although it could easily be modified to work some other way. The return value is a list of properties that failed. Essentially it works by trying to update the entity one property at a time. For each property it removes all properties from the request XML except the one being tested. If the request fails, it assumes that the current property (being the only one we sent) was one of the properties that caused the failure. Start by pasting the following code into your TableServiceContext, then calling TroubleshootInvalidInputError(entity) with the entity that is failing.

      private string mTroubleshootingCurrentProperty = null;

      public List<string> TroubleshootInvalidInputError(TableServiceEntity e)
      {
          WritingEntity += new EventHandler<ReadingWritingEntityEventArgs>(TroubleshootInvalidInputErrorBeforeWrite);
          PropertyInfo[] properties = e.GetType().GetProperties();
          List<string> failedProperties = new List<string>();
          foreach (var property in properties)
          {
              mTroubleshootingCurrentProperty = property.Name;
              try
              {
                  UpdateObject(e);
                  SaveChanges();
              }
              catch (Exception ex)
              {
                  failedProperties.Add(property.Name);
              }
          }

          mTroubleshootingCurrentProperty = null;
          return failedProperties;
      }

      void TroubleshootInvalidInputErrorBeforeWrite(object sender, ReadingWritingEntityEventArgs e)
      {
          if (string.IsNullOrEmpty(mTroubleshootingCurrentProperty))
              return;

          // The XML of the properties already being sent to Azure
          XElement properties = e.Data.Descendants(Meta + "properties").First();
          XName keepName = Data + mTroubleshootingCurrentProperty;
          IEnumerable<XElement> propElements = properties.Elements();

          XElement keepNode = null;

          foreach (var propElement in properties.Elements())
          {
              if (propElement.Name == keepName)
                  keepNode = propElement;
          }

          if (keepNode != null)
          {
              properties.RemoveNodes();
              properties.Add(keepNode);
          }
      }

It does not tell you why it failed, but knowing which property is half the battle. In my case, the problem was that I was trying to upload an entity with a property of a custom class that was Serializable. Normally, I'd been used to the Azure serializer ignoring any public properties that were not supported (such as List, custom classes, etc). However, I added a property based on a custom class I made which was marked as Serializable, so Azure attempted to serialize it and the data service triggered an error.