Need help designing some JUnit tests
in [Java Programming]
|
From: Rhino on 21 May 2010 06:57 Eric Sosman <esosman(a)ieee-dot-org.invalid> wrote in news:ht49d0$i91$1(a)news.eternal-september.org: > On 5/20/2010 4:47 PM, Rhino wrote: >> Eric Sosman<esosman(a)ieee-dot-org.invalid> wrote: >>> [...] >>> For constructors, what you want to test is that they throw >>> exceptions when they're supposed to (e.g., >>> InvalidArgumentException), and that the newly-constructed object >>> satisfies all the invariants it's supposed to. In your no-argument >>> Foo() constructor exceptions seem unlikely (possible, though: think >>> HeadlessException), but you might check `myFoo.getHeight() * >>> myFoo.getWidth() == myFoo.getArea()' or whatever. >>> >> Could you expand on this last paragraph a bit. The part about >> verifying that exceptions get thrown at the appropriate time are fine >> but I'm not clear on what you mean by the invariants that it is >> supposed to satisfy. The last sentence is almost not quite clear. Are >> you saying to find some aspect of the work that is done in the >> constructor and verify that it took place, so that if it is drawing a >> GUI component, that the component exists and has dimensions greater >> that 0 x 0? What if the constructor does very very little - maybe >> just a super() - or even nothing at all? In those cases, is it >> sufficient to just do > > Read the Javadoc for your Foo class, and consider what > properties a newly-constructed Foo instance should satisfy. > Test that they are satisfied -- that is, test that a newly- > constructed Foo complies with its "contract." > Okay, you've given me a broad principle and that is all well and good but I do a lot better with at least one concrete example. It doesn't have to be _my_ example either, just something that resembles something I'm doing. >> if (Foo == null) fail("Constructor failed to instantiate the >> class"); > > Rhino, if you keep on spewing this sort of codecrap I'm going > to shove that horn of yours firmly up the orifice that spews. > I'm really not sure what you mean by "codecrap" here. I assume you're saying that it won't compile but, in fact, this is a line from an actual test case with the class name changed to Foo. Here is the full unit test, WITHOUT the name change, copied and pasted directly from the test case and I assure you that this does compile: public void testGetInstance() { StringUtils stringUtils = StringUtils.getInstance(); if (stringUtils == null) fail("testGetInstance() failed"); } >>> You should test that the factory method behaves as advertised. >>> If it can return null under some circumstances, you should check >>> that it does so when it's supposed to and does not when it isn't. >>> If it returns non-null, the thing returned will necessarily be of >>> the type declared for the factory method -- but not necessarily of >>> that "exact" type, as it might be a subclass or any arbitrary >>> implementation of an interface type. That may make a difference in >>> what you test. >>> >> Can you elaborate on this a bit? Can you show me a simple example of >> a constructor returning a subclass or implementation of an interface? > > No, because a constructor cannot do such a thing. But you seemed > to be talking about a factory method (although it's hard to be sure > from reading your codecrap), Yes, I do in fact mean a factory method. And here are my constructors and getInstance() methods, copied and pasted directly from the source: private StringUtils() { locale = Locale.getDefault(); localizationUtils = LocalizationUtils.getInstance(locale); locMsg = localizationUtils.getResources(locale, MSG_PREFIX); msgFmt.setLocale(locale); } private StringUtils(Locale myLocale) { locale = myLocale; localizationUtils = LocalizationUtils.getInstance(locale); locMsg = localizationUtils.getResources(locale, MSG_PREFIX); msgFmt.setLocale(locale); } public static StringUtils getInstance() { return new StringUtils(); } public static StringUtils getInstance(Locale myLocale) { return new StringUtils(myLocale); } > and a factory method -- any method, in > fact -- can return anything compatible with its declared type. Wasn't > it you who had the problem with SpinnerNumberModel recently, where the > getNumber() method sometimes returned a Short, sometimes a Long, > sometimes something else? > Yes, that's right. Oh, so _that's_ what you meant in your previous reply! Again, when someone answers a question only with a generality, it's never as clear for me as when I also hear a concrete "for instance'. I'm really not being obtuse although I imagine it must seem like it.... >>> I'd say it's unnecessary to test wait() and notify() and other >>> final methods of Object. More generally, it's probably unnecessary >>> to test final methods of any superclass. >>> >>> But equals() is not final, and if the class being tested has >>> its >>> own equals() you should test it. (Note that it's extremely rare to >>> inherit equals() from a superclass unless you're inheriting it all >>> the way from Object undisturbed.) If you were writing tests for >>> Integer, you might test `new Integer(42).equals(new Integer("42"))', >>> for example, and `! new Integer(42).equals(new Integer("-42"))'. >> >> So, in a nutshell, only test the methods of the parent classes if I >> overrode them; otherwise, don't worry about them. That makes sense to >> me! > > There's a subtle point there, a conflict between "black box" > and "clear box" testing. The only way you can *know* that a subclass > inherits a non-final method rather than overriding it is to peek > into the subclass' implementation (either by looking at the source > or by using reflection). But what if somebody comes along next week > and decides to override a method you decided not to test, on the > grounds that it was not overridden? > My theory is rather weak so I'm not really up on the meanings of "black box" and "clear box" testing, let alone the subtle differences between them. Also, I'm not writing the code for anyone but myself right now, except that I _would_ like to get one project's code looking as professional as possible so that I could present it to a prospective client or employer as a portfolio of what I can do. (And then imitate that in future projects as well as gradually retrofit other existing projects with what I've learned). With that in mind, would a reasonable employer/client likely find it acceptable that I just tested the methods I wrote and overrode myself in my classes or are they going to see me as the biggest idiot since the development of COBOL if I fail to observe these subtleties? > One thing you might do is run some of Super's unit tests on Sub > instances. Another might be to include a "sanity check" test in your > Sub, something that reflects on Sub and verifies that the methods > you've chosen not to test are in fact inherited. > > Finally, you've got to realize that unit testing, important as it > is, is not the be-all and end-all of verifying correctness. > I have no problem with that at all. If there are certain things I don't need to cover in unit testing, that's perfectly fine. If you or anyone else reading this could point me a good summary of what is and is not a concern in unit testing, that would be very helpful. Again, I have effectively NO formal training and what much of the on-the-job stuff I have was learned many years ago. That means that my memory of the theory is very incomplete at this point. In short, I don't know what the prevailing theory is on exactly what should be covered by unit testing, acceptance testing, regression testing, et. al. I'm not going to worry about anything beyond unit testing for the moment but if anyone can point me to a general - and hopefully fairly brief and example-laden - discussion of the prevailing theories of testing, that would be very helpful. -- Rhino
From: Lew on 21 May 2010 07:58 On 05/21/2010 06:57 AM, Rhino wrote: >>> if (Foo == null) fail("Constructor failed to instantiate the >>> class"); Eric Sosman wrote: >> Rhino, if you keep on spewing this sort of codecrap I'm going >> to shove that horn of yours firmly up the orifice that spews. Rhino wrote: > I'm really not sure what you mean by "codecrap" here. I assume you're He's referring to the fact that the code you showed will not compile. > saying that it won't compile but, in fact, this is a line from an actual > test case with the class name changed to Foo. Here is the full unit test, Nope. What you should have shown is the *instance* name changed to 'foo'. > WITHOUT the name change, copied and pasted directly from the test case > and I assure you that this does compile: > > public void testGetInstance() { > > StringUtils stringUtils = StringUtils.getInstance(); > if (stringUtils == null) fail("testGetInstance() failed"); > } Rhino, Rhino, Rhino. In the code about which Eric complained, you compared 'Foo', a *class*, to 'null'. That WILL NOT COMPILE! In the code you assure us does compile you compare an instance to 'null'. Of course that compiles. Apples and oranges. You cannot compare a class to 'null', only an instance. Classes and instances are different things. What works for one does not necessarily work for the other. Also, that "actual" code you show neglects the curly braces around the statement body of the 'if'. -- Lew
From: Eric Sosman on 21 May 2010 08:33 On 5/21/2010 7:08 AM, Rhino wrote: > Lew<noone(a)lewscanon.com> wrote in news:ht4ghr$cbl$1(a)news.albasani.net: > >> Rhino wrote: >>>> if (Foo == null) fail("Constructor failed to instantiate the >>>> class"); >> >> Eric Sosman wrote: >>> Rhino, if you keep on spewing this sort of codecrap I'm going >>> to shove that horn of yours firmly up the orifice that spews. >> >> Rhino, the source of Eric's irritation is that despite apologizing >> profusely multiple times for posting uncompilable code, you did it >> again anyway. >> >> Apologies don't help much if you don't correct the behavior. >> > For what it's worth, I never imagined that anyone was actually going to > compile that code. I just provided that (sort-of) code for illustrative > purposes. [...] Nobody was probable to compile the code, not. But a fragment in full load of the errors is not the "illustrative," it is only offuscatorio. In order to extract the meaning from a beautiful insignificant piece of the code, the reading of the person must make the changes and corrections to arrive to something images that you could have in mind and are every probability that will supply some different thing from what has meant and that its ulterior observations therefore will be not at all useful you. (In other words: "Illustrate" your thoughts with garbled utterances, and people may form opinions about your thinking.) -- Eric Sosman esosman(a)ieee-dot-org.invalid
From: Arved Sandstrom on 21 May 2010 08:51 Rhino wrote: > Eric Sosman <esosman(a)ieee-dot-org.invalid> wrote in > news:ht49d0$i91$1(a)news.eternal-september.org: [ SNIP ] >> There's a subtle point there, a conflict between "black box" >> and "clear box" testing. The only way you can *know* that a subclass >> inherits a non-final method rather than overriding it is to peek >> into the subclass' implementation (either by looking at the source >> or by using reflection). But what if somebody comes along next week >> and decides to override a method you decided not to test, on the >> grounds that it was not overridden? >> > My theory is rather weak so I'm not really up on the meanings of "black > box" and "clear box" testing, let alone the subtle differences between > them. Also, I'm not writing the code for anyone but myself right now, > except that I _would_ like to get one project's code looking as > professional as possible so that I could present it to a prospective > client or employer as a portfolio of what I can do. (And then imitate > that in future projects as well as gradually retrofit other existing > projects with what I've learned). With that in mind, would a reasonable > employer/client likely find it acceptable that I just tested the methods > I wrote and overrode myself in my classes or are they going to see me as > the biggest idiot since the development of COBOL if I fail to observe > these subtleties? No word of a lie, just the fact that you're writing tests already gets you some serious points with employers or clients. And the huge majority of people that matter will be perfectly happy with you writing tests for a class that focus on methods that are lexically in the class. That's what I do myself. You perhaps misunderstood Eric - he didn't mean that there is a subtle difference between black box and white box testing. There isn't - those two techniques are quite different. He simply called the specific point he was making subtle. You don't need to be a testing god - people make FT careers out of understanding testing. But you should know what black box and white box testing are, for example. The Wikipedia page on Software Testing is not a bad start, IMO. Just bear in mind that their terminology and definitions are debatable on some specifics, but this doesn't detract from the overall discussion (in particular the first few sentences under Non-Functional Testing are whacked, try to gloss over them). >> One thing you might do is run some of Super's unit tests on Sub >> instances. Another might be to include a "sanity check" test in your >> Sub, something that reflects on Sub and verifies that the methods >> you've chosen not to test are in fact inherited. > >> Finally, you've got to realize that unit testing, important as it >> is, is not the be-all and end-all of verifying correctness. >> > I have no problem with that at all. If there are certain things I don't > need to cover in unit testing, that's perfectly fine. If you or anyone > else reading this could point me a good summary of what is and is not a > concern in unit testing, that would be very helpful. Again, I have > effectively NO formal training and what much of the on-the-job stuff I > have was learned many years ago. That means that my memory of the theory > is very incomplete at this point. In short, I don't know what the > prevailing theory is on exactly what should be covered by unit testing, > acceptance testing, regression testing, et. al. I'm not going to worry > about anything beyond unit testing for the moment but if anyone can point > me to a general - and hopefully fairly brief and example-laden - > discussion of the prevailing theories of testing, that would be very > helpful. You'll end up doing a fair bit of reading and playing with code to get a good handle on testing overall, but it won't take all that long to get a good grip on the basics. I will make a few personal recommendations, which shouldn't be taken as complete. This is based on going on ten years of J2EE work, so depending on what kind of coding you're doing YMMV...some. 1. Code reviews/inspections and static code analysis with tools _are_ a form of testing. Keep them in mind. 2. Integration tests are as important as unit tests. In a J2EE web application these are indispensable - you may have hundreds or thousands of unit tests all pass but still have integration tests not pass. You'll hear a lot of people, myself included, refer to a specific class of integration tests as functional tests - we mean "application functions from the user perspective" when we use "functional" in that regard. Examples of integration/functional tests in a J2EE web app range all the way from ensuring that good things happen in the logic when you click that "Transmogrify" button all the way to doing a complete pass through a use case and making sure that your "Issue Fishing License" logic works in Angler Administrator 2010. Myself I use Selenium IDE/RC in conjunction with the JUnit framework to write these kinds of tests. 3. Code coverage - Huge, IMO. How can you know that your unit tests or integration tests (or even human-tester-driven acceptance tests) are doing any good unless you know how much of the code is actually being exercised? Code coverage is very simple to do, and for starters you can't go wrong investigating Emma or Cobertura. These simply instrument the Java bytecode, such that when the bytecode is executed (by any mechanism) coverage counts by line/branch/method/class/package are written to HTML or XML reports. 4. Carefully consider the issue of test data - test SQL scripts, mock data, in-memory databases (see http://www.mikebosch.com/?p=8) etc. 5. Your tests are themselves defective. The sad truth is that if the core source code you just wrote is riddled with defects, then so probably are your tests. Main take-away here is, be aware that just because all your unit tests pass, some not insignificant percentage of those results are wrong. As a side note, this is where the higher-level layer of integration tests also helps - it can assist in identifying flawed unit tests. HTH, AHS
From: Rhino on 21 May 2010 09:44
Lew <noone(a)lewscanon.com> wrote in news:ht4fub$bj8$1(a)news.albasani.net: > Rhino wrote: >> Actually, my getLocales() method is really just a convenience method >> that massages the results of Locale.getAvailableLocales() itself. >> >> Just to be sure I'm using the term "convenience method" correctly, >> I'm referring to a method I write that uses existing Java API methods >> but that combines several lines of code into one or two. For example, >> since I > > Yep. > Okay, good. At least I know now that I've correctly understood what _that_ term means. One down and a gazillion to go.... ;-) >> prefer my Locales list to be in alphabetical order, I've written >> this: >> >> public Map<String, String> getLocales() { >> >> Locale[] listOfLocales = Locale.getAvailableLocales(); >> >> Map<String, String> locales = new TreeMap<String, String>(); >> for (Locale singleLocale : listOfLocales) { >> locales.put(singleLocale.toString(), singleLocale.getDisplayName >> (locale)); > > Umm, what is 'locale' in this line? I mean, it's obvious that it's a > 'Locale', but what is it? > That 'locale' is an instance variable that the caller can set via a getInstance(locale) method to set the locale for the class. I'm just trying to make my code "multilingual" so that a user who is operating in French or German or whatever will get the output in their language. >> } >> >> return locales; >> } > > Your 'listOfLocales' variable is, perhaps, not necessary. > > That idiom also works if you want to retrieve the Locales themselves > based on name: > > Map <String, Locale> locales = new TreeMap <String, Locale> (); > > for( Locale loc : Locale.getAvailableLocales() ) > { > locales.put( loc.getDisplayName(locale), loc ); > } > > or something like. > I like it :-) I had originally tried to build the map using the locale as the key but ran up against the fact that Locale is not Comparable so I did it the way you see in this email. But making the DisplayName the key and the Locale as the value works fine and is concise to boot so I'll do it your way. >> As such, I don't know how to do a JUnit test on it, specifically how >> to generate an expected result that can be compared to my actual >> result. It seems self-evident that I have to get my expected result >> in a different way than I get the actual result, otherwise, I'm not >> proving anything. > > Unit tests cannot do everything that the class under test does, > otherwise the unit test class would be the class under test. What > unit tests do is test the "happy path" and various corner cases to > provide a high level of certainty that the tested class will correctly > handle the infinite variety of stuff thrown at it in production. In > other words, a unit test is really a sanity check that takes care of > the most likely issues. If a unit test could prevent all possible > errors, we'd never need logging. > Fair enough. > For your case, you might test that the 'Map' has the same number of > entries as 'getLocales()' has elements and that it correctly returns > the right values for some representative keys. > Okay, that's reasonable. Someone else suggested that I verify that the en_US is there too, since that's part of the contract from Locale.getAvailableLocales(). That should be enough for unit testing then, right? > ... >> Or is it the case that such a method CAN'T have its accuracy tested >> in this way and no such attempt should be made? Is it enough to prove >> that the method executes without throwing an exception? > > You should go farther than that. > >>>> -------------------------------- >>>> >>>> Scenario 3 - getInstance() >>>> >>>> Given a hypothetical class named Fuzz where the constructors and >>>> getInstance() methods are: >>>> >>>> private Fuzz() { >>>> // do something >>>> } >>>> >>>> private Fuzz(Locale locale) { >>>> // do something >>>> // initialize instance variable for locale >>>> } >>>> >>>> public getInstance() { > > Lew wrote: >>> Where is your return value? >>> >>> This won't even compile. >>> > > Rhino wrote: >> Sorry, I just hacked that together to save a minute. I probably >> should have copied in a compiled example.... > > Oopsie. (Giggle) > >>>> Would the following be adequate as JUnit tests for the >>>> getInstance() methods? >>>> >>>> public void testGetInstance() { >>>> >>>> Fuzz fuzz = Fuzz.getInstance(); >>>> if (!fuzz instanceof Fuzz) fail("Failed to instantiate Fuzz"); >>>> } >> ... >> So, if the constructor doesn't >> throw any exceptions and is public, you say that I should test that >> "the returned values exists and is not null". > > Non-nullity of the return value should be handled by an 'assert' in > the factory method, and therefore not necessary to test in the unit > test. Existence is already guaranteed by the successful return of the > factory method. > I'm not very familiar with "assert". I'll look into it on my own and add that code. >> What's my best way of doing that? Am I right in assuming that a >> simple >> >> if (Foo != null) > > Ummm, 'Foo' is a class, right? It better be, and therefore that line > will not compile. > >> will cover both of those? > > How about 'assertNotNull( fuzz );'? > That would go in the test case (as opposed to the class) and wouldn't be redundant with the 'assert' you mentioned that should go in the class itself? -- Rhino |