Sunday, December 30, 2012

2.2.12.11 Multiple Invoke with Averages

This release adds the ability to invoke an operation multiple times and reports the averages for each invocation.


This feature comes in handy for smoke-testing code performance improvements. Basically you can invoke any member function N times (the default being once) and monitor the stats to see if a code change you make actually results in any significant performance improvement.

The drop down list includes the pre-assigned values to invoke the function once, twice, 10, 100, 1,000, 10,000, 100,000 or One Million times or you can enter an explicit number for those really hard to track down bugs.

Also available are the average stats for each subsequent call.

Thursday, November 29, 2012

2.2.12.10 Add to Object Pool

This release let's you add object instances to the Object Pool that can be selected later as an argument to other member functions. A very powerful feature that extends the reach of SmokeTest to encompass all member function argument types.

In many scenarios the output from one function is needed as input to another. For simple arguments like strings and numbers you can get away with copy and paste. For complex objects this doesn't work. Now you can save the output from any function to the Object Pool by clicking the "+" button next to the last result.


This button appears next to the result of any member function call (constructors, methods, properties or fields). Clicking the button adds the instance to the Object Pool which is then available as an argument in any subsequent function call that requires an argument of that type (in this case an ArrayList).

Objects added to the Object Pool are available to select as arguments whenever you click on a reference type argument button. This brings up a dialog that let's you create a new object of that type or select one from the Object Pool.


This gives you the full flexibility you need to capture object instances from interfaces or factory methods and use them as arguments to other functions.

Tuesday, October 23, 2012

2.2.12.9 Create Complex Argument Objects

Just released version 2.2.12.9.

This is a major update that adds the ability to create arguments of any reference type or interface from a tree of derived types.

You can also select existing objects from a pool of objects you have created. This makes it possible to use the results of one function as the input to another, a very powerful feature.

Check it out. 

Sunday, September 23, 2012

SmokeTest for .NET Reflector


.net SmokeTest is a Windows application that presents an interactive interface into any .net Assembly allowing you to smoke-test an API via reflection. You can call any member without ever writing a single line code. An indispensible tool for anyone involved in developing, testing and managing software products for Windows.

Now you can leverage the full power of SmokeTest from inside .NET Reflector. Don't just view the code, smoke-test it. The SmokeTest Add-In allows you to interactively:
  • create an instance of a Class.
  • call any Method
  • get and set any Property
  • read and write any Field
  • create, edit and store object instances to pass as arguments.
  • drill-down and smoke-test the returned result.
For a detailed tutorial on how to use .net SmokeTest see my previous Blog posts.

Installation


Download and install the latest version of .net SmokeTest if you haven't already done so.

From Reflector select the Add-Ins menu item and select the "Add" button then navigate to the directory where you installed .net SmokeTest and choose the SmokeTest.Reflector.AddIn.dll file. This should load the SmokeTest Add-In into your Reflector installation.



Using the SmokeTest Add-In


Now when you navigate to a smoke-testable Type in Reflector's Assembly tree (try System.DateTime in mscorlib) you will see a SmokeTest panel appear as shown here.



For a managed type to be smoke-testable it must be a class or struct (ValueType). Generic classes are currently not supported. Interfaces are not smoke-testable for obvious reasons. Also, unlike Reflector, SmokeTest must instantiate the type under test to invoke its member functions and therefore requires all supporting Assemblies and resources to be available to the Assembly under test. This usually means in the same folder as the Assembly under test.

The Basics

So just for fun let's smoke-test the built-in DateTime class. The same procedures apply to any class in the .net hierarchy including of course your own. Navigate to the System.DateTime class under the mscorlib Assembly. The SmokeTest panel should appear when you select the DateTime type as shown above.

Note that the main smoketest panel displays only the static members and constructors of the DateTime class. Of course, until you actually create an instance of a type all you can invoke are the constructors and static members.

Just out of interest click on the Methods, Properties and Fields tabs to see what other static members are available for this class. Note that Properties and Fields will always "get" the current value automatically when you select them and display the result in the preview panel at the bottom. Constructors and Methods require that you invoke the function explicitly.

Now, under the Properties tab select the "Now" property. The current local time will be displayed in the ToString preview tab as a result of invoking the property.

Notice also the "drill-down" button at the bottom right lights-up with the name of the type returned from calling the property, a DateTime in this case. The drill-down button always links to the last result object whether from a Constructor, Method, Property or Field invocation. Clicking the drill-down button will open another SmokeTest window where you can smoke-test all the members of the object instance (not just the static members).

And that's all there is to it. Now try it out with other classes and methods from your own Assemblies or just browse the Microsoft libraries.

See the SmokeTest help page for more information about using the SmokeTest Control.

Sunday, September 16, 2012

Version 2.2.12.8 Released

Summary of changes in this release:

  • Support for Value (struct) types. 
  • Smoketest functionality included in a separate SmokeTestControl.dll.
  • Includes all x86 and x64 versions in the deliverables.
  • Reflector Add-In modified to reference new SmokeTestControl.dll instead of SmokeTest.exe
  • Minor UI improvements to auto-size the argument controls.

See the web page for more details.

Sunday, August 19, 2012

How to use .net SmokeTest

Here is a brief tutorial on using SmokeTest.

.net SmokeTest is a Windows application that presents an interactive interface into any .net Assembly allowing you to smoke-test an API via reflection. You Construct an instance, call a method, set a property or  field of any class Type from a dialog then view or "drill-down" into the result without ever writing a single line code. An indispensible tool for anyone involved in developing, testing and managing software products for Windows.

You can of course load any Assembly for smoke-testing however the default on startup lists all the in-process assemblies. One of these is the SmokeTest application itself which I will use to demonstrate the power and flexibility of .net SmokeTest.

Start the SmokeTest.exe application.

In the namespace tree on the left Navigate to the com.wesleysteiner.smoketest namespace and highlight the AboutBox class. You should see the AboutBox Constructors tab like this:



AboutBox is the managed class that displays the About .net SmokeTest dialog. This tutorial will demonstrate how you can invoke that dialog interactively. The same procedures apply to any class in the .net hierarchy.

The main application screen displays only the static members and constructors of the class-under-test in the panels on the right. Click on the other tabs to see what static members are available. Under the Properties tab you will notice a single static property AssemblyCopyright that returns a string.


Note that Properties and Fields will always "get" the current value automatically when you select it and display the result in the preview panel at the bottom. Constructors and Methods require that you invoke the function explicitly. The results of the last invocation are always displayed in the preview panel as a call to ToString(). Also the number of ticks and milliseconds to invoke the function are displayed on each call. In this case the "Set" button is disabled since this is a read-only property. For writable properties and fields an editable control would appear in the center of the panel where you could explicitly set the value.

So now, back to the AboutBox constructor tab. Select the single constructor and click on the "Create" button to create an instance of the AboutBox class.


Notice the "drill-down" button at the bottom right lights up with the name of the class just created. The drill-down button is common to any return result including built-in types and even Exceptions. Selecting this button will drill-down into the instance just created.

Click the drill-down button now. This will pop-up a smoketest dialog that lists the Methods, Properties and Fields of this instance of the AboutBox class. Now navigate to the ShowDialog method and click the "Call" button.


This will actually call the ShowDialog method of this AboutBox instance popping up the AboutBox dialog as would occur inside an application.


And there it is. Amazing isn't it!

You can also enter arguments for member functions that require them. For example here is the SmokeTest panel for System.IO.FileStream:


The current implementation supports the built-in argument types directly (int, string, bool, etc.) and any concrete reference types. Arguments are listed in the order they appear in the member declaration. Each argument row consists of the identifier name, an editable value control and a Type button. For built-in types and enums you can enter the value directly in the edit control provided. For reference types the program will create an instance of the class by calling it's default constructor if it exists. You can edit this default instance before invoking the member by clicking the "Edit..." button.


That's all there is to it. Now try it out with other classes and methods from your own Assemblies or just browse the Microsoft libraries. Enjoy!


Saturday, August 18, 2012

Introducing .net SmokeTest for Windows

Introducing .net SmokeTest for Windows 

The idea for SmokeTest came to me while working on a project to create an API for clients. It's easy enough to create an API but to test it thoroughly, beyond unit testing, is a bigger task. Generally you need to create a small test app in code to call the function of interest with all type and values of arguments and check and report the return values. That's a lot of repetitive work especially when the API requires "setup" or "login" procedures before they are functional. There has got to be a better way.

Where there's smoke there's fire!
The first improvement was to create a monolithic test application that would let you navigate to any of the implemented functions and run the test suite for that function. Now you only have to do the setup once. The down-side is that you need to add a new test suite for every new API function and decide if it should share the code from an existing test or create a new one, doable but still a lot of work and not guaranteed to be in sync with your latest and greatest API.

Aha, Reflection
It then occurred to me that I could use .net reflection to discover all the namespaces and classes therein then present a tree of member functions that would always be up-to-date. Now we would just need to write a small "plug-in" that could be invoked for the function under test. Hey, now we're getting somewhere. A single app that is in sync with the API at all times and a small plug-in architecture to test each function. After toying with this concept for a short time the idea of creating and maintaining a plug-in architecture seemed daunting and overkill.

Reflected Arguments
The clincher was the realization that even the arguments and argument types for each member function are available via reflection. So now I could leverage reflection to determine the number and types of arguments to each function. The trick was to come up with a user interface that was easy to use and flexible enough to support any and all argument types.


So armed with a couple of weeks of free time on a cruise ship to Hawaii I decided to code the prototype for proof-of-concept. The result of those efforts was the first release of .net SmokeTest for Windows.

This first version only supported simple argument types like int, double and string. For most applications this covers a lot of the API and proved to be very useful indeed. Developers, QA and managers began using it when it was appropriate. It freed up developer's time since they no longer had to create a separate app to demonstrate the functionality. QA was more productive since they could test every edge condition rather than just what the developers decided to include and management started using it to debug support issues. All in all a useful tool across the board.

This is a real hands-on product. The best way to appreciate it's power is to try it out on your favorite Assembly, so go ahead and download it now, it's FREE. There is still a lot of room for features and enhancements. Here you'll find a list of stories I'm working on and some ideas for new features. Let me know if you have any suggestions or any thoughts you have.

Enjoy!