This Question is Answered

1 "helpful" answer available (2 pts)
8 Replies Last post: Nov 13, 2013 6:45 PM by Matt Ellis  
Kozhevnikov Dmitry Newbie 13 posts since
Aug 24, 2012
Currently Being Moderated

Sep 5, 2013 4:52 AM

Howto test a plugin daemon

Hi, I am developing a plug-in, that provide some metrics over the unit-tests. For now it will be a daemon that just highlifhts the tests.  I have assembly with plugin and assebmly with tests(ReSharperTestEnvironmentAssembly in AssemblyInfo). I try to create tests as it suggested in http://confluence.jetbrains.com/display/NETCOM/2.10+Testing+(R8). But this way is not relevant. This code is not working:

private bool IsNunitTestDeclaration(IMethod method)
{
      if (method == null)
      {
             return false;
      }
      return method.HasAttributeInstance(new ClrTypeName("NUnit.Framework.TestAttribute"), false)
             || method.HasAttributeInstance(new ClrTypeName("NUnit.Framework.TestCaseAttribute"), false); }

So I can debug my plugin in other instance of Visual Studio(slowly and inefficient), but I can not test it. I plan to separate plug-in logic that works with unit-test, and use in other ways. So the question is: how i can test the class that accepts the IMethodDeclaration of unit-test and provides metrics snapshot? Can I provide examples of unit tests in my  tests assembly and process it in plug-in tests(howto)? Are there other ways?

Matt Ellis Novice 364 posts since
Jul 5, 2009
Currently Being Moderated
Sep 10, 2013 12:33 PM in response to: Kozhevnikov Dmitry
Re: Howto test a plugin daemon

To use the test support in the SDK, you need to derive from a class called CSharpHighlightingTestBase. If it's .net 4 or 4.5, that would be CSharpHighlightingTestNet4Base or CSharpHighlightingTestNet4Base. You would override RelativeTestPath to point to where your .cs and .gold.cs files live, relative to the test folder, and then implement test methods named after the files and simply calling DoNamedTest2(). In other words, given files CheckForNull.cs and CheckForNull.gold.cs, you would add:

 

[Test]

public void TestCheckForNull()

{

  DoNamedTest2();

}

 

This will run the test, generate output and compare to the gold file. If the gold file doesn't exist, or is different, the test fails. You should see the temp output in the same directory (or possible the temp folder).

 

We only support this type of functional testing rather than unit testing, because most features require a full abstract syntax tree of the file - and that is not something we can mock or fake in a unit test.

Matt Ellis Novice 364 posts since
Jul 5, 2009
Currently Being Moderated
Sep 10, 2013 1:13 PM in response to: Kozhevnikov Dmitry
Re: Howto test a plugin daemon

That's what the base class does for you - it creates an in-memory instance of ReSharper, a solution and a project. It then adds the .cs file you have on disk as a file in the project, and effectively "opens" the file in an in-memory, invisible text control. The normal ReSharper analysis processes run, and the file gets sections highlighted, such as unused variables, errors in syntax, and the highlights from your plugin. The test base class then generates an annotated text file representing the source code and the areas which are highlighted. This file is compared against the gold file, and if it matches, the test passes. If it doesn't match, then your gold file is missing a highlight, or it's in the wrong place, or got the wrong text, or something. You've either got an error in your code, or in your gold file.

 

It's not possible to test the separate logic in a unit test, because you don't have an instance of IMethodDeclaration to pass in. We don't provide any mechanism to create an IMethodDeclaration without going through the process I've just described.

 

What do you mean when you say you have two different environments? What's different about a debug session?

Matt Ellis Novice 364 posts since
Jul 5, 2009
Currently Being Moderated
Sep 11, 2013 1:30 PM in response to: Kozhevnikov Dmitry
Re: Howto test a plugin daemon

Ah, yes, we can do this. The test framework will create a solution and a project with a default set of references, but you can add custom refs, too, such as nunit.framework.dll. The easiest way to do this is to add an attribute to your test that implements ITestLibraryReferencesProvider. You can create your own custom implementation, but it's easier to use TestReferencesAttribute.

 

You use it like this:

 

[TestReferences(TEST_DATA + @"\nunit.framework.dll", TEST_DATA + @"\whatever.dll")]
public void MyTestClass : CSharpHighlightingTestBase
{
  // ...
}

 

You can specify as many references as you want or need to in the constructor, but if you end up with a lot, it might be easier to create a custom class that derives from TestReferencesAttribute, and overrides GetReferences to return these values. Make it easier to reuse.

 

The TEST_DATA above is a string constant defined in BaseTest (which is one of the base classes for the test support). It's equivalent to the string "%TEST_DATA%", which makes the reference "%TEST_DATA%\nunit.framework.dll" - in other words, it builds a path to the dll using the TEST_DATA environment variable. This env var is created by the test framework, and is set to the same folder where you put your .cs and .gold.cs files. So, if you put nunit.framework.dll in the same folder, this attribute will cause the project to have a reference to nunit.framework.dll.

 

Does this help?

Petr Krebs Newbie 5 posts since
Sep 21, 2009
Currently Being Moderated
Nov 12, 2013 10:19 PM in response to: Matt Ellis
Re: Howto test a plugin daemon

Matt,

would you mind adding a sample of the expected test file format? I am not quite sure what the markup should look like for testing highlightings.

 

Thank you.

Matt Ellis Novice 364 posts since
Jul 5, 2009
Currently Being Moderated
Nov 13, 2013 6:45 PM in response to: Petr Krebs
Re: Howto test a plugin daemon

The best thing to do is to write a test and get it to fail - the generated file will show you what it's expecting. It'll look something like this:

 

class Foo {
  public int Bar |{ }|(0)
  public int this[int i] |{ }|(1)
}

---------------------------------------------------------
(0): ReSharper Underlined Error Highlighting: Accessor declaration is missing
(1): ReSharper Underlined Error Highlighting: Accessor declaration is missing

 

Where the location of a highlight is surrounded by pipe symbols - "|" with the index to the highlight in brackets after the closing pipe. The text for the highlight is added to the end of the file.

More Like This

  • Retrieving data ...