Skip a unit test from a Nose2 Plugin
Question:
I’m having trouble actually skipping a unit test from a Nose2 plugin. I am able to mark the test skipped and see the reason in the final result, but the test still runs. This example code should basically skip any test, as long as the plugin is active.
from nose2.events import Plugin
class SkipAllTests(Plugin):
def startTest(self, event):
event.result.addSkip(event.test, 'skip it')
event.handled = True
If I call event.test.skipTest('reason') it actually raises the SkipTest exception like it should, it’s just that the exception isn’t caught by the test runner, it just raises inside of my startTest hook method. Any ideas?
Answers:
I don’t think you can actually stop a test from running with the startTest hook. The nose2 docs suggest using either matchPath or getTestCaseNames to do this. Here’s a working example using matchPath:
from nose2.events import Plugin
class SkipAllTests(Plugin):
configSection = "skipper"
commandLineSwitch = (None, 'skipper', "Skip all tests")
def matchPath(self, event):
event.handled = True
return False
The matchPath docs actually explictly explain how it can be used to stop tests from running:
Plugins can use this hook to prevent python modules from being loaded
by the test loader or force them to be loaded by the test loader. Set
event.handled to True and return False to cause the loader to skip the
module.
Using this method will prevent the test case from ever being loaded. If you want the test to actually show up in the list as skipped, rather than not have it show up in the list of tests at all, you can do a little bit of hackery with the StartTestEvent:
def dummy(*args, **kwargs):
pass
class SkipAllTests(Plugin):
configSection = "skipper"
commandLineSwitch = (None, 'skipper', "Skip all tests")
def startTest(self, event):
event.test._testFunc = dummy
event.result.addSkip(event.test, 'skip it')
event.handled = True
Here, we replace the actual function the test is going to run with a dummy function that does nothing. That way, when the test executes, it no-ops, and then reports that it was skipped.
I ran into the same problem … my solution for this was:
class SkipMe :
"""
Use this module together with nose2 and preferably with the 'with such.A() ...' construct.
Synopsis :
import nose2.tools.such as such
import inspect
import functools
import unittest
with such.A( 'thingy') as it : # Create a SkipMe object.
skipme = SkipMe( 'my_id' ) # You can give this section/chapter a name
skipme = SkipMe() # or just leave it as 'Default'.
@it.has_setup # Integrate to test setup to skip all tests if you like.
def setup() :
skipme.skip_all() # To skip every test in here.
skipme.skip_all( 'for some reason' ) # Same but give a reason in verbose mode.
...
@it.has_test_setup # Recommended to integrate into setup for every test.
def test_setup() :
skipme.skip_if_reason() # Skip test if an overall skip reason was set.
...
@it.should( "just be skipped" )
@skipme.skip_reg( reason='I want it that way' ) # Intentionally skip a single test
def test_skipping():
...
@it.should( "do something basic")
@skipme.skip_reg( skip_all_on_failed=True ) # Register this test. If it fails, skip all consecutive tests.
def test_basic_things() :
...
@it.should( "test something")
@skipme.skip_reg() # Register this test. It will be marked as passed or failed.
def test_one_thing() :
...
@it.should( "test another thing") # Skip this test if previous 'test_one_thing' test failed.
@skipme.skip_reg( func_list=['test_one_thing'] )
def test_another_thing() : # 'skipme.helper' is a unittest.TestCase object.
skipme.helper.assertIs( onething, otherthing, 'MESSAGE' )
...
it.createTests( globals() )
Purpose : Have a convenient way of skipping tests in a nose2 'whith such.A() ...' construct.
As a bonus you have with the 'object.helper' method a 'unittest.TestCase' object.
You can use it to have all the fine assert functions at hand, like assertRaisesRegexp.
Initialize object with:
:param chapter : A string as identifier for a chapter/section.
Defaults:
chapter : 'Default'
Prerequisites: import nose2.tools.such as such, unittest, inspect, functools
Description:
Basically this class has an internal dict that sets and browses for values.
The dict looks like:
{
'IDENTIFYER_1' : { # This is the value of 'chapter'.
'__skip_all__' : BOOLEAN|FUNC_NAME, # Skip all consecutive tests if this is True or a string.
'func1' : BOOLEAN, # Register functions as True (passed) or False (failed).
'func2' : BOOLEAN, # Also skipped tests are marked as failed.
...
},
'IDENTIFYER_1' : { ... }, # This is the domain of another SkipMe object with
... # a different value for 'chapter'
}
It provides a decorator 'object.skip_reg' to decorate test functions to register them to the class,
meaning updating the internal dict accordingly.
Skipped tests are marked as failed in this context.
Skipping all tests of a 'with such.A()' construct:
Integrate it into the setup and the test setup like so:
with such.A( 'thingy') as it :
skipme = SkipMe()
@it.has_setup
def setup() :
skipme.skip_all()
@it.has_test_setup
def test_setup() :
skipme.skip_if_reason()
If you intend to skip all tests or all consecutive tests after a special test failed,
you need only the '@it.has_test_setup' part.
Register tests with the 'skip_reg' method:
Decorate the test functions with the 'object.skip_reg' method under the @it.should decorator.
Example:
with such.A( 'thingy') as it :
skipme = SkipMe()
# Same setup as above
...
@it.should( "Do something")
@skipme.skip_reg() # Just register this function.
@skipme.skip_reg( reason='SOME REASON' ) # Skip this test.
@skipme.skip_reg( func_list=[TEST_FUNCTION_NAMES] ) # Skip test if one function in the list failed.
@skipme.skip_reg( skip_all_on_failed=True ) # Skip all consecutive tests if this fails.
@skipme.skip_reg( func_list=[LIST_OF_TEST_FUNCTIONS], skip_all_on_failed=True ) # Or both.
Example:
import nose2.tools.such as such
import inspect
import functools
import unittest
with such.A( 'thingy' ) as it :
skipme = SkipMe()
@it.has_test_setup
def test_setup() :
skipme.skip_if_reason()
@it.should( "Do something" )
@skipme.skip_reg()
def test_one():
raise
@it.should( "Do another thing" )
@skipme.skip_reg( func_list=[ 'test_one' ] )
def test_two():
pass
@it.should( "just skip" )
@skipme.skip_reg( reason='I want it that way' )
def test_three():
pass
it.createTests( globals() )
# Then run:
nose2 --layer-reporter --plugin=nose2.plugins.layers -v
# Prints:
A thingy
should Do something ... ERROR
should Do another thing ... skipped because of failed: 'test_one'
should just skip ... skipped intentionally because: 'I want it that way'
...
"""
chapter_of = {}
def __init__( self, chapter=None ) :
"""
Initialize a SkipMe object.
:param chapter: If set, must be a string, else it's 'Default'.
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
self.chapter = chapter
self.helper = self.SkipMeHelper() # Set unittest.TestCase object as helper
@classmethod
def set_chapter( cls, chapter=None, func=None, value=None ):
"""
Mark a function of a chapter as passed (True) or failed (False) in class variable 'chapter_of'.
Expands 'chapter_of' by chapter name, function and passed/failed value.
:param chapter: Chapter the function belongs to
:param func: Function name
:param value: Boolean
:return: None
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
if func is None :
raise ValueError( "{0} {1}.{2}: No input for 'func'".format( "ERROR", 'SkipMe', func_name ) )
if not isinstance( func, str ) :
wrong_type = type( func )
raise ValueError( "{0} {1}.{2}: Invalid input for 'func': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( func ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
if not isinstance( value, bool ) :
raise ValueError( "{0} {1}.{2}: No or invalid input for 'value'".format( "ERROR", 'SkipMe', func_name ) )
if chapter not in cls.chapter_of : # If we have this chapter not yet,
cls.chapter_of[ chapter ] = {} # add it and set skip all to false.
cls.chapter_of[ chapter ][ '__skip_all__' ] = False
if func not in cls.chapter_of[ chapter ] : # If we don't have the function yet, add it with value.
cls.chapter_of[ chapter ][ func ] = value
@classmethod
def get_func_state( cls, chapter=None, func_list=None ):
"""
Return function names out of function list that previously failed
:param chapter: The chapter to search for functions
:param func_list: Browse for these function names
:return: List with failed functions. If none found, an empty list.
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
if func_list is None :
raise ValueError( "{0} {1}.{2}: No input for 'func_list'".format( "ERROR", 'SkipMe', func_name ) )
#-------------------------
# Function candidates to check.
# Collect those candidates, that previously returned as failed or skipped.
# Otherwise, return empty list.
if isinstance( func_list, list ) :
func_candidates = func_list
elif isinstance( func_list, str ) :
func_candidates = [ x.strip() for x in func_list.split( ',' ) ]
else:
wrong_type = type( func_list )
raise ValueError( "{0} {1}: Invalid input for 'func_list': '{2}'n"
.format( "ERROR", func_name, str( func_list ) )
+ "{0} Must be list or comma separated string, but was: '{1}'"
.format( "INFO", wrong_type.__name__ ) )
to_return = [] # List of failed functions
if chapter not in cls.chapter_of : # If chapter not found, just return empty list
return to_return
for func in func_candidates : # Otherwise look for each candidate
if func not in cls.chapter_of[ chapter ] : # if it's in the chapter, skip if not.
continue
if not cls.chapter_of[ chapter ][ func ] : # If it's value is False, append it.
to_return.append( func )
return to_return
@classmethod
def mark_chapter_as_skipped( cls, chapter=None, func=None ):
"""
Mark chapter as skipped. Maybe because of a failed function
:param chapter: Which chapter to mark as skipped
:param func: Maybe the failed function that causes this decision
:return: None
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
# Either func is a name or True.
if func :
if not isinstance( func, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'func': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( func ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
else :
func = True
if chapter not in cls.chapter_of : # If we have this chapter not yet,
cls.chapter_of[ chapter ] = {} # add it and set skip all to false.
cls.chapter_of[ chapter ][ '__skip_all__' ] = func
@classmethod
def chapter_marked_skipped( cls, chapter=None ):
"""
Check if a chapter is marked to skip.
:param chapter: The chapter to check
:return: False : Chapter is not marked to be skipped
True : Chapter was intentionally skipped
String : This function was marked with 'skip_all_on_failed=True' and failed.
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
to_return = False
if chapter not in cls.chapter_of :
return to_return
to_return = cls.chapter_of[ chapter ].get( '__skip_all__', False )
return to_return
def skip_reg( self, func_list=None, skip_all_on_failed=None, reason=None ) :
"""
Synopsis :
skipme = SkipMe( 'my_id' )
@skipme.skip_reg()
def some_test_func() :
...
@skipme.skip_reg( func_list=[ LIST_OF_FUNCTIONS_TO_SKIP_IF_FAILED ], skip_all_on_failed=BOOLEAN,
reason=REASON )
def some_other_test_func():
...
Purpose : Decorator to register functions in a SkipMe object and skip tests if necessary
Incoming values :
:param func_list : List or comma separated string with function names.
Skip this test if one of these functions in the list failed or were skipped.
:param chapter : Identifier string that can be used to control skipping more generally.
Default name for chapter is 'Default'.
:param reason : Skip this test in any case and set a string for the reason.
:param skip_all_on_failed : Boolean. If this test fails, mark the current chapter
to skip the rest of tests.
Outgoing results :
:return : Updated class attribute 'chapter_of', maybe skipped test.
Defaults :
chapter : 'Default'
Prerequisites :
inspect, mars.colors.mcp
Description:
Register functions by decorating the functions.
It returns a dict with the function name as key and the function
reference as value.
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
chapter = self.chapter
if isinstance( func_list, list ) :
func_candidates = func_list
elif isinstance( func_list, str ) :
func_candidates = [ x.strip() for x in func_list.split( ',' ) ]
elif func_list is None:
func_candidates = []
else :
wrong_type = type( func_list )
raise ValueError( "{0} {1}: Invalid input for 'func_list': '{2}'n"
.format( "ERROR", func_name, str( func_list ) )
+ "{0} Must be list or comma separated string, but was: '{1}'"
.format( "INFO", wrong_type.__name__ ) )
if reason and not isinstance( reason, str ) :
wrong_type = type( func_list )
raise ValueError( "{0} {1}: Invalid input for 'reason': '{2}'n"
.format( "ERROR", func_name, str( reason ) )
+ "{0} Must be string, but was: '{1}'"
.format( "INFO", wrong_type.__name__ ) )
def inner_skip_reg( func ) :
@functools.wraps( func )
def skip_reg_wrapper( *args, **kwargs ) :
#-------------------------
# First check if the whole chapter was marked as skipped.
# The function either returns:
# True : Means 'skip_all' was set in the beginning (e.g. with @has_setup)
# False : No, chapter is not marked as to be skipped
# Function name : This function was marked with 'skip_all_on_failed' and failed.
skip_reason = self.get_skip_reason()
if skip_reason :
if isinstance( skip_reason, bool ) :
self.helper.skipTest( "chapter '{0}' because it's marked to be skipped".format( chapter ) )
else :
self.helper.skipTest( "chapter '{0}' because of: '{1}'"
.format( chapter, skip_reason ) )
#-------------------------
# Then check if we are just intended to skip by a reason. If so, mark and skip
if reason :
self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
self.helper.skipTest( "intentionally because: '{0}'".format( reason ) )
#-------------------------
# Now see if one of our functions we depend on failed.
# If so, mark our func as failed and skip it.
if func_candidates :
found_failed = self.__class__.get_func_state( chapter=chapter, func_list=func_candidates )
if found_failed :
self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
self.helper.skipTest( "because of failed: '{0}'".format( ', '.join( found_failed ) ) )
#-------------------------
# Now run the test.
# If it fails (assertion error), mark as failed (False), else mark as passed (True)
# If it fails and was marked as 'skip_all_on_failed', mark chapter as to skip all further tests.
try :
result = func( *args, **kwargs )
self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=True )
except Exception as error :
self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
if skip_all_on_failed :
self.__class__.mark_chapter_as_skipped( chapter=chapter, func=func.__name__ )
if error :
raise error
else :
raise
return result
return skip_reg_wrapper
return inner_skip_reg
def get_skip_reason( self ):
chapter = self.chapter
skip_reason = self.__class__.chapter_marked_skipped( chapter=chapter )
return skip_reason
def skip_all( self, reason=None ):
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if reason is not None :
if not isinstance( reason, str ) :
wrong_type = type( reason )
raise ValueError( "{0} {1}: Invalid input for 'reason': '{2}'n"
.format( "ERROR", func_name, str( reason ) )
+ "{0} Must be string, but was: '{1}'".format( "INFO", wrong_type.__name__ ) )
self.__class__.mark_chapter_as_skipped(chapter=self.chapter, func=reason )
def skip_if_reason( self ):
skip_reason = self.get_skip_reason()
if skip_reason:
if skip_reason is True :
reason = "it's marked to be skipped."
else : # Either function or other text.
if skip_reason in self.__class__.chapter_of[ self.chapter ].keys() :
reason = "'{0}' failed.".format( skip_reason )
else :
reason = "'{0}'.".format( skip_reason )
self.helper.skipTest( "chapter '{0}' because: {1}".format( self.chapter, reason ) )
class SkipMeHelper( unittest.TestCase ):
def runTest(self):
pass
SkipMeHelper.maxDiff = None # No limitation in depth while comparing
I’m having trouble actually skipping a unit test from a Nose2 plugin. I am able to mark the test skipped and see the reason in the final result, but the test still runs. This example code should basically skip any test, as long as the plugin is active.
from nose2.events import Plugin
class SkipAllTests(Plugin):
def startTest(self, event):
event.result.addSkip(event.test, 'skip it')
event.handled = True
If I call event.test.skipTest('reason') it actually raises the SkipTest exception like it should, it’s just that the exception isn’t caught by the test runner, it just raises inside of my startTest hook method. Any ideas?
I don’t think you can actually stop a test from running with the startTest hook. The nose2 docs suggest using either matchPath or getTestCaseNames to do this. Here’s a working example using matchPath:
from nose2.events import Plugin
class SkipAllTests(Plugin):
configSection = "skipper"
commandLineSwitch = (None, 'skipper', "Skip all tests")
def matchPath(self, event):
event.handled = True
return False
The matchPath docs actually explictly explain how it can be used to stop tests from running:
Plugins can use this hook to prevent python modules from being loaded
by the test loader or force them to be loaded by the test loader. Set
event.handled to True and return False to cause the loader to skip the
module.
Using this method will prevent the test case from ever being loaded. If you want the test to actually show up in the list as skipped, rather than not have it show up in the list of tests at all, you can do a little bit of hackery with the StartTestEvent:
def dummy(*args, **kwargs):
pass
class SkipAllTests(Plugin):
configSection = "skipper"
commandLineSwitch = (None, 'skipper', "Skip all tests")
def startTest(self, event):
event.test._testFunc = dummy
event.result.addSkip(event.test, 'skip it')
event.handled = True
Here, we replace the actual function the test is going to run with a dummy function that does nothing. That way, when the test executes, it no-ops, and then reports that it was skipped.
I ran into the same problem … my solution for this was:
class SkipMe :
"""
Use this module together with nose2 and preferably with the 'with such.A() ...' construct.
Synopsis :
import nose2.tools.such as such
import inspect
import functools
import unittest
with such.A( 'thingy') as it : # Create a SkipMe object.
skipme = SkipMe( 'my_id' ) # You can give this section/chapter a name
skipme = SkipMe() # or just leave it as 'Default'.
@it.has_setup # Integrate to test setup to skip all tests if you like.
def setup() :
skipme.skip_all() # To skip every test in here.
skipme.skip_all( 'for some reason' ) # Same but give a reason in verbose mode.
...
@it.has_test_setup # Recommended to integrate into setup for every test.
def test_setup() :
skipme.skip_if_reason() # Skip test if an overall skip reason was set.
...
@it.should( "just be skipped" )
@skipme.skip_reg( reason='I want it that way' ) # Intentionally skip a single test
def test_skipping():
...
@it.should( "do something basic")
@skipme.skip_reg( skip_all_on_failed=True ) # Register this test. If it fails, skip all consecutive tests.
def test_basic_things() :
...
@it.should( "test something")
@skipme.skip_reg() # Register this test. It will be marked as passed or failed.
def test_one_thing() :
...
@it.should( "test another thing") # Skip this test if previous 'test_one_thing' test failed.
@skipme.skip_reg( func_list=['test_one_thing'] )
def test_another_thing() : # 'skipme.helper' is a unittest.TestCase object.
skipme.helper.assertIs( onething, otherthing, 'MESSAGE' )
...
it.createTests( globals() )
Purpose : Have a convenient way of skipping tests in a nose2 'whith such.A() ...' construct.
As a bonus you have with the 'object.helper' method a 'unittest.TestCase' object.
You can use it to have all the fine assert functions at hand, like assertRaisesRegexp.
Initialize object with:
:param chapter : A string as identifier for a chapter/section.
Defaults:
chapter : 'Default'
Prerequisites: import nose2.tools.such as such, unittest, inspect, functools
Description:
Basically this class has an internal dict that sets and browses for values.
The dict looks like:
{
'IDENTIFYER_1' : { # This is the value of 'chapter'.
'__skip_all__' : BOOLEAN|FUNC_NAME, # Skip all consecutive tests if this is True or a string.
'func1' : BOOLEAN, # Register functions as True (passed) or False (failed).
'func2' : BOOLEAN, # Also skipped tests are marked as failed.
...
},
'IDENTIFYER_1' : { ... }, # This is the domain of another SkipMe object with
... # a different value for 'chapter'
}
It provides a decorator 'object.skip_reg' to decorate test functions to register them to the class,
meaning updating the internal dict accordingly.
Skipped tests are marked as failed in this context.
Skipping all tests of a 'with such.A()' construct:
Integrate it into the setup and the test setup like so:
with such.A( 'thingy') as it :
skipme = SkipMe()
@it.has_setup
def setup() :
skipme.skip_all()
@it.has_test_setup
def test_setup() :
skipme.skip_if_reason()
If you intend to skip all tests or all consecutive tests after a special test failed,
you need only the '@it.has_test_setup' part.
Register tests with the 'skip_reg' method:
Decorate the test functions with the 'object.skip_reg' method under the @it.should decorator.
Example:
with such.A( 'thingy') as it :
skipme = SkipMe()
# Same setup as above
...
@it.should( "Do something")
@skipme.skip_reg() # Just register this function.
@skipme.skip_reg( reason='SOME REASON' ) # Skip this test.
@skipme.skip_reg( func_list=[TEST_FUNCTION_NAMES] ) # Skip test if one function in the list failed.
@skipme.skip_reg( skip_all_on_failed=True ) # Skip all consecutive tests if this fails.
@skipme.skip_reg( func_list=[LIST_OF_TEST_FUNCTIONS], skip_all_on_failed=True ) # Or both.
Example:
import nose2.tools.such as such
import inspect
import functools
import unittest
with such.A( 'thingy' ) as it :
skipme = SkipMe()
@it.has_test_setup
def test_setup() :
skipme.skip_if_reason()
@it.should( "Do something" )
@skipme.skip_reg()
def test_one():
raise
@it.should( "Do another thing" )
@skipme.skip_reg( func_list=[ 'test_one' ] )
def test_two():
pass
@it.should( "just skip" )
@skipme.skip_reg( reason='I want it that way' )
def test_three():
pass
it.createTests( globals() )
# Then run:
nose2 --layer-reporter --plugin=nose2.plugins.layers -v
# Prints:
A thingy
should Do something ... ERROR
should Do another thing ... skipped because of failed: 'test_one'
should just skip ... skipped intentionally because: 'I want it that way'
...
"""
chapter_of = {}
def __init__( self, chapter=None ) :
"""
Initialize a SkipMe object.
:param chapter: If set, must be a string, else it's 'Default'.
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
self.chapter = chapter
self.helper = self.SkipMeHelper() # Set unittest.TestCase object as helper
@classmethod
def set_chapter( cls, chapter=None, func=None, value=None ):
"""
Mark a function of a chapter as passed (True) or failed (False) in class variable 'chapter_of'.
Expands 'chapter_of' by chapter name, function and passed/failed value.
:param chapter: Chapter the function belongs to
:param func: Function name
:param value: Boolean
:return: None
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
if func is None :
raise ValueError( "{0} {1}.{2}: No input for 'func'".format( "ERROR", 'SkipMe', func_name ) )
if not isinstance( func, str ) :
wrong_type = type( func )
raise ValueError( "{0} {1}.{2}: Invalid input for 'func': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( func ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
if not isinstance( value, bool ) :
raise ValueError( "{0} {1}.{2}: No or invalid input for 'value'".format( "ERROR", 'SkipMe', func_name ) )
if chapter not in cls.chapter_of : # If we have this chapter not yet,
cls.chapter_of[ chapter ] = {} # add it and set skip all to false.
cls.chapter_of[ chapter ][ '__skip_all__' ] = False
if func not in cls.chapter_of[ chapter ] : # If we don't have the function yet, add it with value.
cls.chapter_of[ chapter ][ func ] = value
@classmethod
def get_func_state( cls, chapter=None, func_list=None ):
"""
Return function names out of function list that previously failed
:param chapter: The chapter to search for functions
:param func_list: Browse for these function names
:return: List with failed functions. If none found, an empty list.
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
if func_list is None :
raise ValueError( "{0} {1}.{2}: No input for 'func_list'".format( "ERROR", 'SkipMe', func_name ) )
#-------------------------
# Function candidates to check.
# Collect those candidates, that previously returned as failed or skipped.
# Otherwise, return empty list.
if isinstance( func_list, list ) :
func_candidates = func_list
elif isinstance( func_list, str ) :
func_candidates = [ x.strip() for x in func_list.split( ',' ) ]
else:
wrong_type = type( func_list )
raise ValueError( "{0} {1}: Invalid input for 'func_list': '{2}'n"
.format( "ERROR", func_name, str( func_list ) )
+ "{0} Must be list or comma separated string, but was: '{1}'"
.format( "INFO", wrong_type.__name__ ) )
to_return = [] # List of failed functions
if chapter not in cls.chapter_of : # If chapter not found, just return empty list
return to_return
for func in func_candidates : # Otherwise look for each candidate
if func not in cls.chapter_of[ chapter ] : # if it's in the chapter, skip if not.
continue
if not cls.chapter_of[ chapter ][ func ] : # If it's value is False, append it.
to_return.append( func )
return to_return
@classmethod
def mark_chapter_as_skipped( cls, chapter=None, func=None ):
"""
Mark chapter as skipped. Maybe because of a failed function
:param chapter: Which chapter to mark as skipped
:param func: Maybe the failed function that causes this decision
:return: None
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
# Either func is a name or True.
if func :
if not isinstance( func, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'func': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( func ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
else :
func = True
if chapter not in cls.chapter_of : # If we have this chapter not yet,
cls.chapter_of[ chapter ] = {} # add it and set skip all to false.
cls.chapter_of[ chapter ][ '__skip_all__' ] = func
@classmethod
def chapter_marked_skipped( cls, chapter=None ):
"""
Check if a chapter is marked to skip.
:param chapter: The chapter to check
:return: False : Chapter is not marked to be skipped
True : Chapter was intentionally skipped
String : This function was marked with 'skip_all_on_failed=True' and failed.
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if chapter is None :
chapter = 'Default' # Set default chapter for convenience
if not isinstance( chapter, str ) :
wrong_type = type( chapter )
raise ValueError( "{0} {1}.{2}: Invalid input for 'chapter': '{3}'n"
.format( "ERROR", 'SkipMe', func_name, str( chapter ) )
+ "{0} Must be string, but was: {1}".format( "INFO", wrong_type.__name__ ) )
to_return = False
if chapter not in cls.chapter_of :
return to_return
to_return = cls.chapter_of[ chapter ].get( '__skip_all__', False )
return to_return
def skip_reg( self, func_list=None, skip_all_on_failed=None, reason=None ) :
"""
Synopsis :
skipme = SkipMe( 'my_id' )
@skipme.skip_reg()
def some_test_func() :
...
@skipme.skip_reg( func_list=[ LIST_OF_FUNCTIONS_TO_SKIP_IF_FAILED ], skip_all_on_failed=BOOLEAN,
reason=REASON )
def some_other_test_func():
...
Purpose : Decorator to register functions in a SkipMe object and skip tests if necessary
Incoming values :
:param func_list : List or comma separated string with function names.
Skip this test if one of these functions in the list failed or were skipped.
:param chapter : Identifier string that can be used to control skipping more generally.
Default name for chapter is 'Default'.
:param reason : Skip this test in any case and set a string for the reason.
:param skip_all_on_failed : Boolean. If this test fails, mark the current chapter
to skip the rest of tests.
Outgoing results :
:return : Updated class attribute 'chapter_of', maybe skipped test.
Defaults :
chapter : 'Default'
Prerequisites :
inspect, mars.colors.mcp
Description:
Register functions by decorating the functions.
It returns a dict with the function name as key and the function
reference as value.
"""
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
chapter = self.chapter
if isinstance( func_list, list ) :
func_candidates = func_list
elif isinstance( func_list, str ) :
func_candidates = [ x.strip() for x in func_list.split( ',' ) ]
elif func_list is None:
func_candidates = []
else :
wrong_type = type( func_list )
raise ValueError( "{0} {1}: Invalid input for 'func_list': '{2}'n"
.format( "ERROR", func_name, str( func_list ) )
+ "{0} Must be list or comma separated string, but was: '{1}'"
.format( "INFO", wrong_type.__name__ ) )
if reason and not isinstance( reason, str ) :
wrong_type = type( func_list )
raise ValueError( "{0} {1}: Invalid input for 'reason': '{2}'n"
.format( "ERROR", func_name, str( reason ) )
+ "{0} Must be string, but was: '{1}'"
.format( "INFO", wrong_type.__name__ ) )
def inner_skip_reg( func ) :
@functools.wraps( func )
def skip_reg_wrapper( *args, **kwargs ) :
#-------------------------
# First check if the whole chapter was marked as skipped.
# The function either returns:
# True : Means 'skip_all' was set in the beginning (e.g. with @has_setup)
# False : No, chapter is not marked as to be skipped
# Function name : This function was marked with 'skip_all_on_failed' and failed.
skip_reason = self.get_skip_reason()
if skip_reason :
if isinstance( skip_reason, bool ) :
self.helper.skipTest( "chapter '{0}' because it's marked to be skipped".format( chapter ) )
else :
self.helper.skipTest( "chapter '{0}' because of: '{1}'"
.format( chapter, skip_reason ) )
#-------------------------
# Then check if we are just intended to skip by a reason. If so, mark and skip
if reason :
self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
self.helper.skipTest( "intentionally because: '{0}'".format( reason ) )
#-------------------------
# Now see if one of our functions we depend on failed.
# If so, mark our func as failed and skip it.
if func_candidates :
found_failed = self.__class__.get_func_state( chapter=chapter, func_list=func_candidates )
if found_failed :
self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
self.helper.skipTest( "because of failed: '{0}'".format( ', '.join( found_failed ) ) )
#-------------------------
# Now run the test.
# If it fails (assertion error), mark as failed (False), else mark as passed (True)
# If it fails and was marked as 'skip_all_on_failed', mark chapter as to skip all further tests.
try :
result = func( *args, **kwargs )
self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=True )
except Exception as error :
self.__class__.set_chapter( chapter=chapter, func=func.__name__, value=False )
if skip_all_on_failed :
self.__class__.mark_chapter_as_skipped( chapter=chapter, func=func.__name__ )
if error :
raise error
else :
raise
return result
return skip_reg_wrapper
return inner_skip_reg
def get_skip_reason( self ):
chapter = self.chapter
skip_reason = self.__class__.chapter_marked_skipped( chapter=chapter )
return skip_reason
def skip_all( self, reason=None ):
func_name = inspect.stack()[ 0 ][ 3 ] # This function Name
if reason is not None :
if not isinstance( reason, str ) :
wrong_type = type( reason )
raise ValueError( "{0} {1}: Invalid input for 'reason': '{2}'n"
.format( "ERROR", func_name, str( reason ) )
+ "{0} Must be string, but was: '{1}'".format( "INFO", wrong_type.__name__ ) )
self.__class__.mark_chapter_as_skipped(chapter=self.chapter, func=reason )
def skip_if_reason( self ):
skip_reason = self.get_skip_reason()
if skip_reason:
if skip_reason is True :
reason = "it's marked to be skipped."
else : # Either function or other text.
if skip_reason in self.__class__.chapter_of[ self.chapter ].keys() :
reason = "'{0}' failed.".format( skip_reason )
else :
reason = "'{0}'.".format( skip_reason )
self.helper.skipTest( "chapter '{0}' because: {1}".format( self.chapter, reason ) )
class SkipMeHelper( unittest.TestCase ):
def runTest(self):
pass
SkipMeHelper.maxDiff = None # No limitation in depth while comparing