forked from testng-team/testng
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGES.txt
1788 lines (1547 loc) · 109 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Current
Fixed: GITHUB-2792: JUnitTestClass sets XmlTest as null when running JUnit 4 Tests using TestNG (Krishnan Mahadevan)
Fixed: GITHUB-2844: Deprecate support for running Spock Tests (Krishnan Mahadevan)
Fixed: GITHUB-550: Weird @BeforeMethod and @AfterMethod behaviour with dependsOnMethods (Krishnan Mahadevan)
Fixed: GITHUB-893: TestNG should provide an Api which allow to find all dependent of a specific test (Krishnan Mahadevan)
New: Added .yml file extension for yaml suite files, previously only .yaml was allowed for yaml (Steven Jubb)
Fixed: GITHUB-141: regular expression in "dependsOnMethods" does not work (Krishnan Mahadevan)
Fixed: GITHUB-2770: FileAlreadyExistsException when report is generated (melloware)
Fixed: GITHUB-2825: Programmatically Loading TestNG Suite from JAR File Fails to Delete Temporary Copy of Suite File (Steven Jubb)
Fixed: GITHUB-2818: Add configuration key for callback discrepancy behavior (Krishnan Mahadevan)
Fixed: GITHUB-2819: Ability to retry a data provider in case of failures (Krishnan Mahadevan)
Fixed: GITHUB-2308: StringIndexOutOfBoundsException in findClassesInPackage - Surefire/Maven - JDK 11 fails (Krishnan Mahadevan)
Fixed: GITHUB:2788: TestResult.isSuccess() is TRUE when test fails due to expectedExceptions (Krishnan Mahadevan)
Fixed: GITHUB-2800: Running Test Classes with Inherited @Factory and @DataProvider Annotated Non-Static Methods Fail (Krishnan Mahadevan)
New: Ability to provide custom error message for assertThrows\expectThrows methods (Anatolii Yuzhakov)
Fixed: GITHUB-2780: Use SpotBugs instead of abandoned FindBugs
Fixed: GITHUB-2801: JUnitReportReporter is too slow
Fixed: GITHUB-2807: buildStackTrace should be fail-safe (Sergey Chernov)
Fixed: GITHUB-2830: TestHTMLReporter parameter toString should be fail-safe (Sergey Chernov)
Fixed: GITHUB-2798: Parallel executions coupled with retry analyzer results in duplicate retry analyzer instances being created (Krishnan Mahadevan)
7.6.1
Fixed: GITHUB-2761: Exception: ERROR java.nio.file.NoSuchFileException: /tmp/testngXmlPathInJar-15086412835569336174 (Krishnan Mahadevan)
7.6.0
Fixed: GITHUB-2741: Show fully qualified name of the test instead of just the function name for better readability of test output.(Krishnan Mahadevan)
Fixed: GITHUB-2725: Honour custom attribute values in TestNG default reports (Krishnan Mahadevan)
Fixed: GITHUB-2726: @AfterClass config method is executed for EACH @Test method when parallel == methods (Krishnan Mahadevan)
Fixed: GITHUB-2752: TestListener is being lost when implenting both IClassListener and ITestListener (Krishnan Mahadevan)
New: GITHUB-2724: DataProvider: possibility to unload dataprovider class, when done with it (Dzmitry Sankouski)
Fixed: GITHUB-217: Configure TestNG to fail when there's a failure in data provider (Krishnan Mahadevan)
Fixed: GITHUB-2743: SuiteRunner could not be initial by default Configuration (Nan Liang)
Fixed: GITHUB-2729: beforeConfiguration() listener method should be invoked for skipped configurations as well(Nan Liang)
Fixed: assertEqualsNoOrder for Collection and Iterators size check was missing (Adam Kaczmarek)
Fixed: GITHUB-2709: Testnames not working together with suites in suite (Martin Aldrin)
Fixed: GITHUB-2704: IHookable and IConfigurable callback discrepancy (Krishnan Mahadevan)
Fixed: GITHUB-2637: Upgrade to JDK11 as the minimum JDK requirements (Krishnan Mahadevan)
Fixed: GITHUB-2734: Keep the initial order of listeners (Andrei Solntsev)
Fixed: GITHUB-2359: Testng @BeforeGroups is running in parallel with testcases in the group (Anton Velma)
Fixed: Possible StringIndexOutOfBoundsException in XmlReporter (Anton Velma)
Fixed: GITHUB-2754: @AfterGroups is executed for each "finished" group when it has multiple groups defined (Anton Velma)
7.5
Fixed: GITHUB-2701: Bump gradle version to 7.3.3 to support java17 build (ZhangJian He)
Fixed: GITHUB-2646: Streamline Logging Across TestNG (Krishnan Mahadevan)
Fixed: GITHUB-2658: Inheritance + dependsOnMethods (Krishnan Mahadevan)
Fixed: GITHUB-2664: Order for DependsOnGroups has changed after TestNg 7.4.0 (Krishnan Mahadevan)
Fixed: GITHUB-2501: TestNG 7.4.0 throws an exception "sun.net.www.protocol.file.FileURLConnection cannot be cast to java.net.HttpURLConnection" when xml file contain "ENTITY SYSTEM" grammer (Krishnan Mahadevan)
Fixed: GITHUB-2693: TestNG ignores 'dataproviderthreadcount' CLA (Krishnan Mahadevan)
Fixed: GITHUB-2685: TestInvoker should clear Thread.interrupted flag before calling ITestListeners (Roman Morskyi)
Fixed: GITHUB-2684: AfterGroups config annotation does not consider retries for tests (Roman Morskyi)
Fixed: GITHUB-2689: Yaml parser: implement loadClasses flag (Dzmitry Sankouski)
Fixed: GITHUB-2676: NPE is triggered when working with ITestObjectFactory (Krishnan Mahadevan)
Fixed: GITHUB-2674: Run onTestSkipped for each value from data provider (Krishnan Mahadevan)
Fixed: GITHUB-2672: Log real stacktrace when test times out. (cdalexndr)
Fixed: GITHUB-2669: A failed retry with ITestContext will lose the ITestContext. (Nan Liang)
Fixed: GITHUB-2643: assertEquals(Set,Set) now ignores ordering as it did before. (Elis Edlund)
Fixed: GITHUB-2653: Assert methods requires casting since TestNg 7.0 for mixed boxed and unboxed primitives in assertEquals.
Fixed: GITHUB-2229: Restore @BeforeGroups and @AfterGroups Annotations functionality (Krishnan Mahadevan)
Fixed: GITHUB-2563: Skip test if its data provider provides no data (Krishnan Mahadevan)
Fixed: GITHUB-2535: TestResult.getEndMillis() returns 0 for skipped configuration - after upgrading testng to 7.0 + (Krishnan Mahadevan)
Fixed: GITHUB-2638: "[WARN] Ignoring duplicate listener" appears when running .xml suite with <listeners> and <suite-files> (Krishnan Mahadevan)
Fixed: GITHUB-1297: Passed configuration methods appear in testng-failed.xml, when failure was after passed test (Dzmitry Sankouski)
New: Decouple configuration unit tests from main suite (Dzmitry Sankouski).
Fixed: GITHUB-2536: Problems with Nested Test Classes (Krishnan Mahadevan)
Fixed: GITHUB-2558:Make IExecutionListener, ITestListener, IInvokedMethodListener, IConfigurationListener, ISuiteListener finish method with reverse order (dianny)
Fixed: GITHUB-2532: Apply commandline switches for suites in jar files (Dzmitry Sankouski).
Fixed: GITHUB-2558: Make IExecutionListener, ITestListener, IInvokedMethodListener, IConfigurationListener, ISuiteListener execute in the order of insertion (Krishnan Mahadevan)
Fixed: GITHUB-2611: Config Failures not included in testng-failed.xml when its part of a different test class (Krishnan Mahadevan)
Fixed: GITHUB-2613: Ignored Tests are not retrieved for a mixed test class (test with enabled, disabled and ignored test method) (Krishnan Mahadevan)
Fixed: GITHUB-849: Performance improvement by fixing hashCode (testn & Vladimir Sitnikov)
Fixed: GITHUB-2570: Use Guice injector for instantiate IRetryAnalyzer (Krishnan Mahadevan)
Fix: use proper instances for beforeClass callback when different instances collide on hash codes
Fixed: Fix parallel and configfailurepolicy parsing in tr_TR locale
Fixed: Wrong results from GuiceBasedObjectDispenser when there's Object#hashCode collision
Changed: GITHUB-2564: Source code is split into several modules for better modularity in the future (for now only a combined jar is released as it was before)
Added: GITHUB-2564: Added license files as META-INF/LICENSE.txt within the released jar
Test: GITHUB-2564: Added pax-exam-based OSGi test to verify the manifest
Changed: GITHUB-2564: Migrated the build to Gradle 7.0.2
Fixed: GITHUB-2576: Guice 5.0 drops no-aop variant, so TestNG should probably upgrade and avoid no-aop dependency (Nan Liang)
Fixed: GITHUB-2566: Reporter#getOutput(ITestResult tr) uses Map.get(tr.hashCode()) which might result in surprising results (Krishnan Mahadevan)
Fixed: GITHUB-2565: Dataprovider only supporting a raw type for Iterator return type (Krishnan Mahadevan)
Fixed: GITHUB-2557: Flaky test: ThreadAffinityTest#testThreadAffinity (Krishnan Mahadevan)
Fixed: GITHUB-2567: MethodHelper#CANONICAL_NAME_CACHE is never reset, so it could result in a memory leak (Krishnan Mahadevan)
Fixed: GITHUB-2540: assertEquals(Collection) need check order (Stuart Marks & Julien Herr)
Fixed: GITHUB-2360: Groovy 3 internal generated methods are detected as test methods (Ian Springer)
Fixed: GITHUB-2522: TestNG 7.4.0 Can not skip test through listener (Nan Liang)
Fixed: GITHUB-2529: Link to testng.xml in CONTRIBUTING.md file was dead
Fixed: GITHUB-2521: The method has a separate string with 'invocation-number' parameters for each failure (does not group) in 'testng-failed.xml' file for DataProvider + Factory (Pavel Sakharchuk)
Fixed: GITHUB-2426: New feature TestNG - getFactoryMethodParamsInfo on ConfigurationMethod (Krishnan Mahadevan)
Fixed: GITHUB-2517: Factory data-provider parameters not displayed in 'testng-failed.xml' file (Pavel Sakharchuk)
Fixed: GITHUB-279: Guice dependency injection into listeners and reporters (Krishnan Mahadevan)
Fixed: GITHUB-2504: Data provider data cannot be populated to onTestStart hook when BeforeMethod is failed (Krishnan Mahadevan)
Fixed: GITHUB-2489: Hierarchical base- and test-class @AfterClass methods out of order using groups (Krishnan Mahadevan)
Fixed: GITHUB-2493: Avoid NPE from TextReporter execution when a dataprovider method provides null (baflQA)
Fixed: GITHUB-2483: Asymmetric not equals (cdalexndr)
Fixed: GITHUB-2486: assertSame/assertNotSame broken after GITHUB-2296 (Vitalii Diravka)
Fixed: GITHUB-2490: assertNotEquals returns fast when argument is null, not calling equals(Object) (Anindya Roy)
Fixed: GITHUB-2500: Mention in assertEqualsNoOrder doc that arrays are not compared deeply (Marcono1234)
Fixed: GITHUB-2544: TestNG Retry Fails on complex data-provider arguments (Anindya Roy)
7.4.0
New : GITHUB-2459: Support configurable start time - emailable report (Barry Evans)
Fixed: GITHUB-2467: XmlTest does not copy the xmlClasses during clone (C.V.Aditya)
Fixed: GITHUB-2469: Parameters added in XmlTest during AlterSuiteListener not available in SuiteListener (C.V.Aditya)
Fixed: GITHUB-2296: Fix for assertEquals not working for sets as order is not guaranteed. (Prashant Maroti)
Fixed: GITHUB-2465: Fix bux where Strings.join returns empty String
Fixed: GITHUB-1632: throwing SkipException sets iTestResult status to Failure instead of Skip (Julien Herr & Krishnan Mahadevan)
New : GITHUB-2456: Add onDataProviderFailure listener (Krishnan Mahadevan)
Fixed: GITHUB-2445: NPE in FailedReporter.java With Tests Created in Factory (Arham Jain)
Fixed: GITHUB-2428: Configuration methods have the same test class instance when @Factory is being used (Nan Liang)
Fixed: GITHUB-2440: Fixed an issue when case timeout returned an incorrect exception and effect the next other test case (Yao Ma)
New: GITHUB-2407: Adds "overrideIncludedMethods" to the global config as a command-line argument, which excludes explicitly included test methods if they belong to any excluded groups (Nikhil Suri)
Fixed: GITHUB-2432: Rework MethodInheritance.fixMethodInheritance to "soft" dependencies (Krishnan Mahadevan)
Fixed: GITHUB-2429: Seggregate Dependency Injection out as a clear implementation (Krishnan Mahadevan)
Fixed: GITHUB-2435: getParameterIndex() always return 0 in test listener
Fixed: GITHUB-2406: TestNG 7.3.0 transitive vulnerability CVE-2020-11022 and CVE-2020-11023 due to JQuery 3.4.1 (Krishnan Mahadevan)
Fixed: GITHUB-2405: Regression: Using TestNG via Maven breaks when optional Guice dependency is unavailable (Krishnan Mahadevan)
Fixed: GITHUB-2427: Guice module (suite parent-module and test module) configure() method is called multiple times (Jacek Centkowski)
Fixed: GITHUB-2419: TestNG JUnit reports are not valid if system output contains XML tags (Lorenzo Orsatti)
Fixed: GITHUB-188: suite parallel="methods" does not work when there are multiple <test> tags in the testng.xml (Krishnan Mahadevan)
Fixed: GITHUB-346: When a method is annotated with both BeforeGroups and AfterGroups only AfterGroup is executed (Krishnan Mahadevan)
Fixed: GITHUB-2403: Suite.xml files attempt to make web request when suite references standard TestNG DTD using HTTP (Krishnan Mahadevan)
New: GITHUB-2385: Make @Listeners can work for implemented interfaces and Inherited class (Nan Liang)
Fixed: GITHUB-2053: MethodHelper.collectAndOrderMethods() Hangs when Parallel Instance and dependsOnGroups (Krishnan Mahadevan)
Fixed: GITHUB-2400: BeforeClass/Method (and AfterClass/Method) configuration methods that override default methods are invoked multiple times (Krishnan Mahadevan)
Fixed: GITHUB-2396: @Ignore on method level doesn't work as expected (Krishnan Mahadevan)
Fixed: GITHUB-2382: TestNG version should be specified in MANIFEST.MF (Krishnan Mahadevan)
Fixed: GITHUB-2096: 7.0.0-beta6 memory issues (regression) (Krishnan Mahadevan)
Fixed: GITHUB-2355: TestNG creates multiple Guice Module Instances (Krishnan Mahadevan)
Fixed: GITHUB-2374: Add file name to the warning message (Krishnan Mahadevan)
Fixed: GITHUB-2321: -Dtestng.thread.affinity=true do not work when running multiple instance of test in parallel (Nan Liang)
Fixed: GITHUB-2363: JS error when switching theme (Krishnan Mahadevan)
Fixed: GITHUB-2361: No way to enforce @Test(singleThreaded = true) when test defined in base class (Krishnan Mahadevan)
Fixed: GITHUB-2343: Injectors are not reused when they share the same set of modules (Krishnan Mahadevan)
Fixed: GITHUB-2346: ITestResult attributes are null when retrieved by Listener onTestStart if test fails at BeforeMethod (Krishnan Mahadevan)
Fixed: GITHUB-2357: TestNG 7.3.0 transitive dependencies
7.3.0
Fixed: GITHUB-2328: Add ability to get test method for which configuration method was called (Krishnan Mahadevan)
Fixed: GITHUB-2327: Parameters not present on skipped Test (Eric Kubenka)
Fixed: GITHUB-2232: Null Pointer Exception in ConfigInvoker.setMethodInvocationFailure (Krishnan Mahadevan)
Fixed: GITHUB-2312: IAnnotationTransformer called multiple time Discrepancy between 6.x and 7.x (Krishnan Mahadevan)
New: GITHUB-2315: TextReporter console output does not nicely print native array data parameters (James Sassano)
Fixed: GITHUB-2301: Add support for object-based reporter configurations (Scott Babcock)
New: Deprecate org.testng.ReporterConfig (Julien Herr)
Fixed: GITHUB-2300: Vulnerable Dependency: Please upgrade JCommander to 1.75 or above (Krishnan Mahadevan)
Fixed: GITHUB-2182: Removed exception catching as valid behaviour
Fixed: GITHUB-2273: Use SPI to load Guice modules (Bartosz Popiela)
Fixed: GITHUB-2280: Prevent Retry from happening endlessly (Paweł Nadolski)
Fixed: GITHUB-553: Better error message when dealing with classes having both Constructor and Factory methods (Krishnan Mahadevan)
Fixed: GITHUB-2267: RetryAnalyzer is not set properly and consistent when using dataprovider (Eric Kubenka)
Fixed: GITHUB-2266: Support Test retries via Callbacks (Krishnan Mahadevan)
Fixed: GITHUB-2209: @Before and @After are not executed as expected when a combination of class and method level grouping is applied (Krishnan Mahadevan)
Fixed: GITHUB-2259: Missing configuration for ServiceLoader Listeners (Krishnan Mahadevan)
Fixed: GITHUB-2257: Facilitate retry of configuration methods via call backs (Krishnan Mahadevan)
Fixed: GITHUB-2223: testng 7.1.0 java.lang.ClassNotFoundException: com.google.inject.Stage (Krishnan Mahadevan)
Fixed: GITHUB-217: Configure TestNG to fail when all tests are skipped (Krishnan Mahadevan)
Fixed: GITHUB-2255: Ensure test method parameters are visible in BeforeMethod config method (Krishnan Mahadevan)
Fixed: GITHUB-2251: NullPointerException at test with timeOut (Krishnan Mahadevan)
Fixed: GITHUB-2249: Not abstract super-classes mess up test run order (Sergii Kim)
Fixed: GITHUB-2195: NPE Using groups and @Before/@AfterMethod with alwaysRun and dependsOnMethods (Tomas & Julien Herr)
Fixed: GITHUB-2238: Parameter values should be overridable from JVM arguments (Krishnan Mahadevan)
Fixed: GITHUB-2231: Incorrect hierarchy in testng.xml file created programmatically and failed to run. (Krishnan Mahadevan)
Fixed: GITHUB-2235: expectedExceptions mismatch because of wrapping (Krishnan Mahadevan)
Fixed: GITHUB-2220: ITestListener's methods get called multiple times for one test, when @Listeners annotation is used in multiple test classes (Krishnan Mahadevan)
Fixed: GITHUB-2211: assertEquals for Map sometimes does not use provided message (Krishnan Mahadevan)
Fixed: GITHUB-1632: throwing SkipException sets iTestResult status to Failure instead of Skip (Krishnan Mahadevan)
Fixed: GITHUB-2207: Allow dependency injector factory to be configured via args (dmikhievich)
Fixed: GITHUB-2193: Ignore local url for DTD security check (Li.Zhao & Julien Herr)
Fixed: GITHUB-1968: Upgrade to Gradle 6.0
7.1.0
New : GITHUB-2199: Allow users to provide their own Injector for Dependency Injection (Krishnan Mahadevan)
Fixed: GITHUB-2180: Write scenario details for Retried tests (Devendra Raju K)
Fixed: GITHUB-2124: JUnit Report should contain the output of org.testng.Reporter (Krishnan Mahadevan)
Fixed: GITHUB-2171: Ability to embed attachments and make them available on TestNG XML report (Krishnan Mahadevan)
Fixed: GITHUB-2152: Multiple Test Groups Causing @BeforeMethod and @AfterMethod to be called multiple times for a single test (Krishnan Mahadevan)
Fixed: GITHUB-2172: Suite summary report table issue with EmailableReporter2.java (Devendra Raju K)
Fixed: GITHUB-2148: TestNG - configfailurepolicy=“continue” is not working for retried test (Krishnan Mahadevan)
Fixed: GITHUB-2163: Test is executed infinite number of times when the data provider returns a new object (Krishnan Mahadevan)
Fixed: GITHUB-2157: NullPointerException occurs when a Retried test has an exception in DataProvider (Krishnan Mahadevan)
Fixed: GITHUB-2150: Upgraded jQuery from 1.7.1 to 3.4.1 to resolve reported prototype pollution vulnerability
Fixed: GITHUB-2149: Handle NoClassDefFoundError when classloader fails to load a class
New: GITHUB-2111: Provide an interceptor for Data Provider (Krishnan Mahadevan)
Fixed: GITHUB-1709: @Ignore doesn't work when used on child class, and parent has multiple @Test methods (Krishnan Mahadevan)
New: GITHUB-2118: Default assertion message (Jiong Fu)
Fixed: GITHUB-2080: Wrong text for assertTrue (Jiong Fu)
Fixed: GITHUB-2078: Xml to Yaml does not capture the dependency definition in generated Yaml File (Jiong Fu)
7.0.0
Fixed: GITHUB-2137: Issue with Priority with too low values (Krishnan Mahadevan)
Fixed: GITHUB-2138: Factory calls are not intercepted by IAnnotationTransformer (Krishnan Mahadevan)
Fixed: GITHUB-2121: SoftAssert: allow custom message (dr29bart)
Fixed: GITHUB-2110: NPE is thrown when running with Thread affinity (Krishnan Mahadevan)
Fixed: GITHUB-2069: JUnit TestSuite not handled correctly in Reports (Krishnan Mahadevan)
Fixed: GITHUB-2075: Thread interrupt flag persists between test methods (Krishnan Mahadevan)
Fixed: GITHUB-2074: Thread Interrupted when using expectedException (Krishnan Mahadevan)
Fixed: GITHUB-2802: some methods' javadoc in ISuiteListener and ITestListener is misleading (Yehui Wang)
Fixed: GITHUB-2061: java.util.ConcurrentModificationException after registration of SuiteListener at the runtime (Krishnan Mahadevan)
Fixed: GITHUB-2055: It's not possible to register a new ITestListener at the runtime (Krishnan Mahadevan)
Fixed: GITHUB-1035: @BeforeClass not executed in parallel when parallel="instances" (Krishnan Mahadevan)
Fixed: GITHUB-2043: IConfigurationListener is not executed and IDataProviderListener is not added to the list of the listeners if a new listener is added at the runtime (Krishnan Mahadevan)
Fixed: GITHUB-1835: Configurable ThreadPoolExecutor (Krishnan Mahadevan)
Fixed: GITHUB-1691: Support reading data provider information from Class level @Test annotation (Krishnan Mahadevan)
Fixed: GITHUB-326: When group-by-instances is set to true the instances created by @Factory does not run in parallel (Krishnan Mahadevan)
Fixed: GITHUB-1930: Running testng-failed.xml correctly runs failed test in child class but incorrectly runs all tests in base class (Krishnan Mahadevan)
Fixed: GITHUB-1987: Retrieve the data provider method's reference from the test method (Krishnan Mahadevan)
Fixed: GITHUB-1976: Add a proper message when types are missing during annotation parsing (Krishnan Mahadevan)
Fixed: GITHUB-2000: ThreadPoolExecutor ConcurrentModificationException (Krishnan Mahadevan)
Fixed: GITHUB-2022: Suite.xml files make web request when suite uses DTD over HTTPS (Krishnan Mahadevan)
Fixed: GITHUB-2017: Don't use child injectors for Guice tests (Joe Barnett)
New: GITHUB-2003: Add to @Test interface fields IDs and issues (Krishnan Mahadevan)
Fixed: GITHUB-2009: Test Timeout not respected in parallel="methods" mode (Krishnan Mahadevan)
Fixed: GITHUB-2008: Preserve parameters in each class and not all of the test parameters
Fixed: GITHUB-1981: Fixes NPE in Assert.assertEquals when an array contains null (Maneesh MS)
New: Upgrade to gradle5
New: Added a method in Assertion class to allow downstream TestNG consumers to override the error message (Ryan Laseter)
Fixed: GITHUB-165: @AfterGroups is not executed when group member fails or is skipped (Krishnan Mahadevan)
Fixed: GITHUB-118: @BeforeGroups only called if group is specified explicitly (Krishnan Mahadevan)
Fixed: GITHUB-182: Inherited test methods do not get expected group behavior (Krishnan Mahadevan)
Fixed: GITHUB-1988: Add Automatic-Module-Name to MANIFEST.MF (Krishnan Mahadevan)
Fixed: GITHUB-1985: Custom "IMethodSelector" implementation doesn't filter methods properly (Krishnan Mahadevan)
Fixed: GITHUB-993: Handle null test names from ITest implementation (Krishnan Mahadevan)
Fixed: GITHUB-1942: assertDeepEquals takes long time to evaluate size mismatch (Kumaran Bharathan)
Fixed: GITHUB-1967: IInvokedMethod ITestResult status set as -1 for test methods skipped due to config failure (Krishnan Mahadevan)
Fixed: GITHUB-1952: Provide a TestNGListener that can be invoked when a test fails due to a timeout (Krishnan Mahadevan)
Fixed: GITHUB-1953: TestNG throws a misleading error when @Factory method returns empty array. (Krishnan Mahadevan)
Fixed: GITHUB-1946: Retry analyzer does not work properly when coupled with a data provider (Krishnan Mahadevan)
Fixed: GITHUB-1924: Testclass instantiation fails when both no-arg constructor and factory method present (Krishnan Mahadevan)
Fixed: GITHUB-1935: Wrong text for assertEquals (Krishnan Mahadevan)
Fixed: GITHUB-1931: [NPE] Reporter org.testng.reporters.jq.Main failed (Krishnan Mahadevan, Oleg Shaburov)
New: Remove raw type warnings in ConversionUtils
Fixed: GITHUB-1480: Parallel=methods not working when tests have different priorities set (Micah Lapping-Carr)
Fixed: GITHUB-1041: Factory data-provider parameters not displayed in test-result (Krishnan Mahadevan)
Fixed: GITHUB-1901: The overall reported Test time for suite containing parallel tests should be max(tests_times) (Krishnan Mahadevan)
Fixed: GITHUB-1893: Streamline invocation of "init" method within TestResult to be private (Krishnan Mahadevan)
Fixed: GITHUB-1892: Configurable InvokedMethodListener (Krishnan Mahadevan)
Fixed: GITHUB-435: Apply <packages> at suite level to all tests (Siegmar Alber)
Fixed: GITHUB-1870: Fix Ambiguous behavior of IInvokedMethodListener by clarifying javadocs(Krishnan Mahadevan)
Fixed: GITHUB-1878: Provide visibility into the actual method (config/test) that caused a downstream test to be skipped (Krishnan Mahadevan)
Fixed: GITHUB-1883: Establish Listener invocation order as tests for documentation purposes (Krishnan Mahadevan)
Fixed: TestNG doesn't use the searchable loaders for JUnit tests (Igor Ignatev)
Fixed: GITHUB-1880: @AfterGroups(alwaysRun = true) is not called if there is an exception in @BeforeGroups (Krishnan Mahadevan)
Fixed: GITHUB-1874: Prevent circular dependency error when suite includes different methods from same class (Krishnan Mahadevan)
Fixed: GITHUB-1863: IMethodInterceptor will be invoked twice when listener implements both ITestListener and IMethodInterceptor via eclipse execution way.(Bin Wu)
Fixed: GITHUB-1865: testngXmlPathInJar- cannot be cleared when vm terminate(Yehui Wang)
Fixed: GITHUB-1850: Parser returns a wrong structure when parent suite has duplicate child suites (Chao Qin)
Fixed: GITHUB-1803: Added new methods for comparing float and double arrays with delta (Atul Agrawal)
Fixed: GITHUB-1661: Fixed Assert logic for two dimensional arrays (Atul Agrawal)
Fixed: GITHUB-1734: Added unit tests for NaN, Max, Min, Positive and Negative infinity (Atul Agrawal)
Fixed: GITHUB-1740: Bumped up artifact version of dependencies for java 8 support (Atul Agrawal)
Fixed: GITHUB-1834: Ensure group dependency defined via suite xml is considered (Krishnan Mahadevan)
New : Expose Graph Visualisation representation for users to build real-time debugging tools (Krishnan Mahadevan)
Fixed: GITHUB-574: String cannot be cast to Integer if log property is set in maven pom.xml (Krishnan Mahadevan)
Fixed: GITHUB-1402: TestNG reporting - Quickly identify Retry-d Tests (Krishnan Mahadevan)
Fixed: GITHUB-1697: Need ability to mark "failed but up for retry" tests differently from "skipped" tests (Krishnan Mahadevan)
Fixed: GITHUB-1810: Nullpointer exception when running TestNG tests from CMD (Krishnan Mahadevan)
Fixed: GITHUB-1590: Started configuration method has wrong status and end time during execution (Krishnan Mahadevan)
Fixed: GITHUB-298: Avoid Javascript errors in function name when suite name has special characters (Krishnan Mahadevan)
Fixed: GITHUB-1048: Fix validation errors in TestNG CSS to adhere to CSS level 3 + SVG (Krishnan Mahadevan)
Fixed: GITHUB-341: TestNG does not respect "-parallel classes" when running with a jar file (Krishnan Mahadevan)
Fixed: GITHUB-1790: IAnnotationTransformer transform method is not called for all test classes annotated with @Test (Krishnan Mahadevan)
Fixed: GITHUB-1787: Method level parameters are missing when using Yaml generation utilities (Krishnan Mahadevan)
New : TestNG now guarantees thread-affinity for methods that use either preserve-order (or) dependsOnMethods (Krishnan Mahadevan)
Fixed: GITHUB-1773: Parallel="classes" executes methods of test class in different threads (Krishnan Mahadevan)
Fixed: GITHUB-1185: DependsOnMethods made parallel class use several threads (Krishnan Mahadevan)
Fixed: GITHUB-1173: Parallel="classes" executes methods of test class in different threads (Krishnan Mahadevan)
Fixed: GITHUB-1066: Regression is in priority. It broke parallel mode (Krishnan Mahadevan)
Fixed: GITHUB-1050: Parallel classes runs methods from one class in different threads, interleaves two classes in one thread (Krishnan Mahadevan)
Fixed: GITHUB-89: parallel="classes" is not forcing test methods from the same testClass to be run in the same thread as it is suposed to (Krishnan Mahadevan)
Fixed: GITHUB-1719: successPercentage does not work correctly for tests with dataProvider (Krishnan Mahadevan)
Fixed: GITHUB-1241: Streamline Retry Analyzer usage when same test is run multiple times (Krishnan Mahadevan)
Fixed: GITHUB-1777: ITestListener.onTestStart() not called after fail or skip from @BeforeMethod (Krishnan Mahadevan)
Fixed: GITHUB-1778: SoftAssert#fail swallows actual root cause (Krishnan Mahadevan)
Fixed: GITHUB-1665: Failed test after rerun would impact next test case result (Yehui Wang)
Fixed: GITHUB-1770: Factory annotation does not work with Parameters annotation on method constructor (Krishnan Mahadevan)
Fixed: GITHUB-1767: Prevent NPE when XmlTest accessed without a proper XmlSuite (Krishnan Mahadevan)
Fixed: GITHUB-1766: Testng generates a lot of temp folders like testngXmlPathInJar- (Andrey)
Fixed: GITHUB-1759: core interfaces refactoring to support default methods (Sergey Korol)
Fixed: GITHUB-1753: TestResult for an SKIP test lose attributes contributed by @BeforeMethod's or @AfterMethod's (Krishnan Mahadevan)
Fixed: GITHUB-1756: ITest .getTestName() doesn't return the actual test name for skipped methods (Krishnan Mahadevan)
Fixed: GITHUB-1602: beforeInvocation method don't get called for skipped tests (Krishnan Mahadevan)
Fixed: GITHUB-549 and GITHUB-780: Introduce onlyForGroups attribute for @BeforeMethod and @AfterMethod (Sergei Tachenov)
Fixed: GITHUB-1745: Support native injection for @Factory methods (Krishnan Mahadevan)
Fixed: GITHUB-1746: Make InvokedMethodNameListener thread-safe, fixing occasional build failures (Sergei Tachenov)
Fixed: GITHUB-1538: Dependent methods don't get invoked when a failed test method is retried via a RetryAnalyser (Krishnan Mahadevan)
Fixed: GITHUB-426: firstTimeOnly ignored for threadPoolSize > 1 (Krishnan Mahadevan)
Fixed: GITHUB-466: JUnitReportReporter always converts the system time into GMT (Krishnan Mahadevan)
Fixed: GITHUB-1723: Standardize timezone reference in XML reports (Krishnan Mahadevan)
Fixed: GITHUB-489: Incorrect timezone info in <test-method> element of testng-results.xml (Krishnan Mahadevan)
Fixed: GITHUB-1735: IExecutionListener.onStart() running twice when used as annotation (Krishnan Mahadevan)
New : Removed deprecated methods across TestNG (Krishnan Mahadevan)
Fixed: GITHUB-1726: Allow user-defined method interceptors to have the last say in method re-ordering (Krishnan Mahadevan)
Fixed: GITHUB-564: Support using @Optional on @Test method parameters to use null values (Krishnan Mahadevan)
New : Upgrade TestNG to start needing at-least JDK8 (Krishnan Mahadevan)
Fixed: GITHUB-1637: Remove dependency on a pre-created jar for running unit tests for JarFileUtils (Krishnan Mahadevan)
Fixed: GITHUB-1710: Inefficient DynamicGraph causes hang with only 100 tests. (Steve Prentice)
Fixed: GITHUB-1716: Potential NPE in jq.Main reporter (Krishnan Mahadevan)
Fixed: GITHUB-1709: @Ignore doesn't work when used on child class, and parent has @Test methods (Krishnan Mahadevan)
Fixed: GITHUB-1706: Retrying of methods fail when test method involves native injection (Krishnan Mahadevan)
New : Removed deprecated attributes from annotations (Julien Herr)
New : Support all JSR-223 compatible script engine
Fixed: GITHUB-1827: TestNG does not throw an error when a test class does not have a proper constructor (Julien Herr & Krishnan Mahadevan)
Fixed: Annotated default methods of indirectly implemented interfaces should still be called (Ilya Korobitsyn)
New : GITHUB-2105: Include assertEquals(Map, Map) into Assert class
6.14.3
Fixed: GITHUB-1077: TestNG cannot handle load (Aheiss)
Fixed: GITHUB-1081: group-by-instances with test dependencies causes instantiation of tests to exponentially slow (Aheiss)
Fixed: GITHUB-1700: Test ignored if @BeforeMethod in base class fails for another test class (Krishnan Mahadevan)
Fixed: GITHUB-1694: @BeforeGroups executed multiple times when tests run in parallel, once if not parallel (Krishnan Mahadevan)
Fixed: GITHUB-1688: @Ignore annotation on base class doesn't ignore tests in child classes (Krishnan Mahadevan)
Fixed: GITHUB-1687: NullPointerException is thrown when beanshell evaluates to null (Krishnan Mahadevan)
6.14.2
Fixed: GITHUB-1674: beanshell methodselector applied at suite level is ignored (Krishnan Mahadevan)
Fixed: GITHUB-1668: "Invalid Method Selector" exception triggered when using suite level beanshell methodselectors (Krishnan Mahadevan)
Fixed: GITHUB-1659: New line characters are removed from stack traces in testng-results.xml (Krishnan Mahadevan)
Fixed: GITHUB-1503: Consider adding a -n (no execute) option (Krishnan Mahadevan)
Fixed: GITHUB-1648: Depends on method is not respected on the sequential run on second test that extends same base testClass (Krishnan Mahadevan)
Fixed: GITHUB-1636: Parallel test run is not working in 6.13.1 (Krishnan Mahadevan)
New : GITHUB-1634: Make "-xmlpathinjar" support <suite-files> (Yehui Wang)
New : GITHUB-1634: Make "-testnames" find tests from Multi-level parent-child suites (Yehui Wang)
Fixed: GITHUB-1641: The time for test-method is 8 hours ahead of the the time for suite/class in testng-result xml file (Krishnan Mahadevan)
Fixed: GITHUB-1649: @Test annotated methods cannot inject java.lang.reflect.Method (Krishnan Mahadevan)
Fixed: GITHUB-1625: Null fields in parallel method tests (Krishnan Mahadevan)
Fixed: GITHUB-1605: Research the usefulness of the JVM argument "experimental" (Krishnan Mahadevan)
New : GITHUB-1631: data provider class name injection into Factory meta-data (Sergey Korol)
6.13.1
No functional changes. Released with newer version JCommander (1.72.0)
6.13
Fixed: GITHUB-1619: ConcurrentHashMap doesn't secure insertion order.(Yehui Wang)
Fixed: GITHUB-1616: Test cases with priority and dependsOnGroups dependencies, execution order is chaos. (Yehui Wang)
Fixed: GITHUB-1613: The constructor removed from TestRunner would stop Eclipse working.(Yehui Wang)
Fixed: GITHUB-1600: Updated in afterInvocation() testResult.status is not used in willRetry condition (Krishnan Mahadevan)
Fixed: GITHUB-1598: Injected types parameter and optional parameters cannot be used together. (Yehui Wang)
Fixed: GITHUB-1594: Cannot filter by "testnames" when suite xml is a suite of suites (Krishnan Mahadevan)
Fixed: GITHUB-1584: Can't run tests from IDEA (Krishnan Mahadevan)
Fixed: GITHUB-1589: TestNGAntTask should be consistently using the Ant Log API for writing log messages (Krishnan Mahadevan)
Fixed: GITHUB-1587: TestNG can not guarantee the ExecutionListener Instance as singleton(Yehui Wang)
Fixed: GITHUB-217: exception in DataProvider doesn't fail test run (Krishnan Mahadevan)
Fixed: GITHUB-987: Parameters threadCount and parallel doesn't work with maven (Krishnan Mahadevan)
Fixed: GITHUB-1472: Optimize DynamicGraph.getUnfinishedNodes (Krishnan Mahadevan & Nathan Reynolds)
Fixed: GITHUB-1566: Invalid XML characters in Params in testng-results.xml (Krishnan Mahadevan)
Fixed: GITHUB-1554: @Parameters and parameter injection not wroking when used on the same method (Krishnan Mahadevan)
Fixed: GITHUB-990: NullPointerExceptions after a superclass configuration method fails with configfailurepolicy="continue" (Krishnan Mahadevan)
Fixed: GITHUB-461 : Annotate annotations with @Documented (Krishnan Mahadevan)
Fixed: GITHUB-778 : XmlSuite toXml() does NOT add the suite time-out property (Krishnan Mahadevan)
Fixed: GITHUB-1029: Issue with getting XmlTest from test method (Krishnan Mahadevan)
Fixed: GITHUB-212: Enable support for providing a URI as suite file location (Krishnan Mahadevan)
Fixed: GITHUB-161 : Provide a way to customize SAXParserFactory implementation (Krishnan Mahadevan)
Fixed: GITHUB-1455: Configure XML output of XmlSuite (Krishnan Mahadevan)
Fixed: GITHUB-1465: Failure policy CONTINUE handling is broken for tests that are skipped in @BeforeMethod method (Krishnan Mahadevan)
Fixed: GITHUB-1533: Duplicate child suites get added when working with parent/child suite scenario (Krishnan Mahadevan)
Fixed: GITHUB-949: dependsOnMethods with alwaysRun = true and inheritance fails to find method (Krishnan Mahadevan)
Fixed: GITHUB-1519: Possibility to retry a test until it FAILED (Krishnan Mahadevan)
Fixed: GITHUB-980: TestNG run inherited method twice (Krishnan Mahadevan)
Fixed: GITHUB-1409: Regression: on expectedExceptionsMessageRegExp expected and actual messages are not printed (Krishnan Mahadevan)
Fixed: GITHUB-1517: TestNG exits with a zero when there are configuration failures (Krishnan Mahadevan)
New : GITHUB-1490: Add a listener for data provider interception (Krishnan Mahadevan)
Fixed: GITHUB-1456: Remove/Warn support of constructor with String param (Krishnan Mahadevan)
Fixed: GITHUB-1509: Improve error message when data provider returns a null value (Krishnan Mahadevan)
Fixed: GITHUB-1507: TestNG runs all methods when filtering via <include> fails (Krishnan Mahadevan)
Fixed: GITHUB-1493: Wrong exception msg when timeout on test (Krishnan Mahadevan)
Fixed: GITHUB-328: Attempt to fix unnecessary execution of @Factory-ctors (@beverage & Julien Herr)
Fixed: GITHUB-1384: Huge performance issue between 6.5.2 and 6.11 (Denis Bazhenov)
New: Remove Serializable (Julien Herr)
Fixed: GITHUB-1496: If method contains "$", run only one method, all methods will be run (@JF-Rabbit & Julien Herr)
Fixed: GITHUB-1220: Recognize annotations on default methods in implemented interface
New: GITHUB-861: Add @Ignore annotation which disables all tests in a class or a package (Julien Herr)
6.12
Fixed: GITHUB-1484: Remove irrelevant "targets" for TestNG annotations (Krishnan Mahadevan)
Fixed: GITHUB-1405: Skip considering main() method when @Test used at class level (Krishnan Mahadevan)
Fixed: GITHUB-799: @Factory with dataProvider changes order of iterations (Krishnan Mahadevan & Julien Herr)
New: Enhance XML Reporter to be able to customize the file name (Krishnan Mahadevan)
Fixed: GITHUB-1417: Class param injection is not working with @BeforeClass (Krishnan Mahadevan)
Fixed: GITHUB-1440: Improve error message when wrong params on configuration methods (Krishnan Mahadevan)
Fixed: GITHUB-1433: Missing encoding for emailable reports (Shaburov Oleg)
Fixed: GITHUB-1430: Cannot load class from file XXX when using with ant and classfileset (Olivier Mourez)
Fixed: GITHUB-1394: Optimize ClassHelper.getAvailableMethods() to exclude Object class(Nathan Reynolds & Krishnan Mahadevan)
Fixed: GITHUB-1396: Order established by IMethodInterceptor not honored when running with parallel='instances' (Ryan Scott)
Fixed: GITHUB-1287: Parallel (methods) execution with dependsOn running in unexpected order (Kevyn Reinholt)
Fixed: GITHUB-1362: Ensure AfterGroups methods get executed when involving Method Interceptors (Krishnan Mahadevan)
Fixed: GITHUB-765: Skip invocation of bridged methods (Krishnan Mahadevan)
New: Enhance TestNGAntTask to be customizable (Denys Kurylenko)
New: Make EmailableReporter2 W3C Compliant[XHTML 1.1] (Chris Rankin)
Fixed: GITHUB-1336: (parallel=‘classes’) not working when coupled with priority (Krishnan Mahadevan)
Fixed: GITHUB-1365: Be able to override default XML parser (@ChristiKh & Julien Herr)
Fixed: GITHUB-1360: TestNG does not distinguish between methods of different priorities (Krishnan Mahadevan)
Fixed: GITHUB-1144: Add Class and Constructor as legal native dependency injection (Guillaume Juillot)
Fixed: GITHUB-1380: Circular dependencies may fail in parallel (Julien Herr)
Fixed: GITHUB-1400: TestNG, Multiple duplicate listener warnings on implementing multiple listener interfaces (@bipo1980 & Nick Tan)
Fixed: GITHUB-1426: @AfterMethod(alwaysRun = true) is not getting called if we have exception in @BeforeMethod (@dipak-pawar)
Fixed: GITHUB-128: Using Object[] and Method as parameters for a test in a certain order yields an IllegalArgumentException, citing a type mismatch (@leef590 & Julien Herr,Krishnan Mahadevan)
Fixed: GITHUB-1393: Revert commit 50d534a to allow fail a test from onTestStart method
Fixed: GITHUB-1461: TestNG not getting garbage collected (@kiru)
6.11
2017/02/27
Fixed: GITHUB-1351: FailurePolicy failing with YAML (Steven Zaluk & Julien Herr)
New: The name of all TestNG threads follow "TestNG-<thread type>-<number>" pattern (Julien Herr)
Fixed: GITHUB-1339: Alter ClassHelper to use Maps instead of Lists for extracting methods (Krishnan Mahadevan)
Fixed: GITHUB-1338: @BeforeGroups method is run on a wrong instance (Pavel Vetokhin & Julien Herr)
Fixed: GITHUB-1332: Make EmailableReport name configurable (Krishnan Mahadevan)
Fixed: GITHUB-1297: testng-failed.xml includes setup and tearDown (before and after annotations) of passed tests
(Krishnan Mahadevan)
Fixed: GITHUB-1319: ITestResult#getInstance() returns null in IConfigurationListener implementation (Krishnan Mahadevan)
New: Skipped methods are not supposed to be executed (Julien Herr)
Fixed: GITHUB-1197: Ability to dynamically set the status and exception of a test via ITestResult (Anthony Nguyen)
Fixed: GITHUB-1302: When 'parallel' is set to 'classes', ConcurrentModificationException can be thrown(Jianhua Li)
Fixed: GITHUB-772: Severe thread contention while running large test with parallel methods (Shaburov Oleg)
Fixed: GITHUB-1307: TestNGException when using an anonymous class in Factory (Mike Cowan)
Fixed: GITHUB-1298: ITestResult injection is failing in BeforeMethod method (Krishnan Mahadevan)
Fixed: GITHUB-1293: Beanshell based execution does not work any more (Krishnan Mahadevan)
Fixed: GITHUB-1262: Testcases out of order in XML file in junitreport folder when using testng (Krishnan Mahadevan)
Fixed: GITHUB-116 : BaseTestMethod does respect general contract of Comparable (Testo Nakada)
Fixed: GITHUB-1265: JUnit Reporter includes redundant ignored methods (Krishnan Mahadevan)
Fixed: GITHUB-1266: JUnit Reporter produces a wrong number of total test methods (Krishnan Mahadevan)
Fixed: GITHUB-1257: Group parameter not applying on included <suite-files>
Fixed: GITHUB-1284: Listeners on the child suites are not applied (Vimalraj Selvam)
New: GITHUB-1313: Add Java9 as test environment on Travis (Julien Herr)
Fixed: GITHUB-1296: Configuration listeners run multiple times (@mikimrozowski & Julien Herr)
Fixed: GITHUB-1300: Add deep assertions to assert on array values (array reference by default) (Jordan Zimmerman & Julien Herr)
6.10
2016/11/28
Fixed: GITHUB-551: Failed configuration method always has 0 execution time (dr29bart)
Fixed: GITHUB-1250: Testng-failed.xml is getting test level parameters into suite level parameters (Krishnan Mahadevan)
Fixed: GITHUB-1046: Provide a mechanism to customize a test method name for reporting (Krishnan Mahadevan)
Fixed: GITHUB-1211: Include disabled/ ignored test methods in JUnit reports (Krishnan Mahadevan)
Fixed: GITHUB-1213: Include "ignored" test count in testng-results.xml (Krishnan Mahadevan)
Fixed: GITHUB-674: Enrich Test method skips due to configuration failures with throwable data (Krishnan Mahadevan)
Fixed: GITHUB-1240: Enrich the test results showing mechanism in Travis CI (Krishnan Mahadevan)
Fixed: GITHUB-1232: Prevent TestNG from adding duplicate instances of the same listener (Krishnan Mahadevan)
Fixed: GITHUB-1170: Fixing the test DataProviderTest.shouldNotThrowConcurrentModification (Krishnan Mahadevan)
Fixed: GITHUB-1231: Make IExecutionListener implementation be the last reporter call before JVM exit(Krishnan Mahadevan)
Fixed: GITHUB-1227: Prevent multiple instances of same Reporter from being injected into TestNG (Krishnan Mahadevan)
Fixed: GITHUB-1165: Better message to user when param injection is not good (Krishnan Mahadevan)
Fixed: GITHUB-1228: Control stacktrace levels in XmlReports via a JVM configuration (Krishnan Mahadevan)
Fixed: GITHUB-1203: Add flush to BufferedWriter; fixes incomplete XML reports (Nathan Bruning)
Fixed: GITHUB-1181: Fix MethodMatcherException: Data provider mismatch (Krishnan Mahadevan)
Fixed: GITHUB-1107: TestNG does not report/print/log throwables in data providers (Krishnan Mahadevan)
Fixed: GITHUB-1186: NullPointerException in JUnit reporter when used with Spock (Ian Robertson & Julien Herr)
Fixed: GITHUB-1180: NullPointerException on getting excluded/included groups (Krishnan Mahadevan)
Fixed: GITHUB-1064: Incorrect logging of parallel mode of a test
Fixed: GITHUB-1178: Halt execution when invalid testname is provided. (Krishnan Mahadevan)
Fixed: GITHUB-1139: DataProvider could support Object[] as a valid return type (Julien Herr)
Fixed: GITHUB-1182: Cannot run multiple @Factory-annotated methods in the same class (Ian Donovan & Julien Herr)
New: Hierarchy on order features (from less important to more important): groupByInstance, preserveOrder, priority, dependsOnGroups, dependsOnMethods
Fixed: GITHUB-1156: test execution dependant upon class name order and fails with TestNGException: No free nodes found (@t-weil & Julien Herr)
Fixed: GITHUB-1221: ConcurrentModificationException in TextReporter (Nick Tan)
Fixed: testng-eclipse/issues/298: IConfigurationListener was not loaded when running Test in Eclipse Plugin (@jmcgrail & Nick Tan)
New: Resources for test reports (header, images) now live in a sub directory org/testng to remove conflicts with other similar files in the class loader.
6.9.13.6
2016/09/23
6.9.13.5 (Bad release)
2016/09/23
6.9.13.4 (Bad release: Wrong internal version)
2016/09/22
6.9.13.3 (Ok, but too many direct dependencies)
2016/09/22
6.9.13.2 (Bad release: Wrong internal version)
2016/09/20
6.9.13.1 (Bad release: fat jar)
2016/09/19
6.9.13 (Bad release: JDK8 target only)
2016/09/16
New: code improvement in order to calculate key for dependency map. Dependency map will use methodQualifiedName as key provided by ITestNGMethod (Chirag Jayswal)
Fixed: GITHUB-1105: Test skipped instead failed if incorrect enum value (Liza Ivanova & Julien Herr)
Fixed: GITHUB-1111: XMLReporter crashes if a test parameter is exactly "]]>" (Łukasz Rekucki & Julien Herr)
Fixed: GITHUB-1108 @BeforeGroups called twice (Krishnan Mahadevan)
Fixed: GITHUB-1112 XmlInclude.getDescription returns null always (Krishnan Mahadevan)
Fixed: GITHUB-1090 Inconsistent handling "preserve-order" on suite/test level (Michal Domagala & Julien Herr)
Fixed: GITHUB-1085 Remove Guava dependency (Erik C. Thauvin & Julien Herr)
Fixed: GITHUB-1084 Using deprecated addListener methods should not register many times (Anna Kozlova & Julien Herr)
Fixed: GITHUB-447 Copy test parameters instead of storing a reference (Huagang Li & Julien Herr)
Fixed: GITHUB-174: NPE when parsing xml where <suite> has <groups> (Peter Stout & Julien Herr)
Fixed: GITHUB-918: NullPointerException on loading XmlSuites programmatically (@ispitkovskyi & Julien Herr)
Fixed: GITHUB-689: <groups> at <suite> level not applied to <suite-files> (@kunal546 & Julien Herr)
Fixed: GITHUB-740: More than one execution even when success with DataProvider and IRetryAnalyzer (Sergio Sacristán)
Fixed: GITHUB-877: Retries don't work correctly with DataProvider tests (Simonas Tvirbutas)
Fixed: GITHUB-1103: Add ...junit.ArrayAsserts.assertArrayEquals(boolean[], boolean[]) (Jonathan Halterman & Julien Herr)
Fixed: GITHUB-1122: Use the default value for preserve-order (Guillaume Guillot & Julien Herr)
Fixed: GITHUB-1022: Non static methods from external data providers are not working without @Guice (Sourav Chandra & Julien Herr)
Fixed: GITHUB-1130: IClassListener should only be instantiated once (Guillaume Guillot & Julien Herr)
Fixed: GITHUB-1131: IObjectFactory not being called for factory test instances with constructor-injected data provider (Scott McClure & Julien Herr)
New: GITHUB-1083: Factory supports indices (Julien Herr)
Fixed: GITHUB-148: 'Run Failed Test' doesn't run the proper tests after 2nd rerun when DataProvider is used (@akracheva & Julien Herr)
6.9.12
2016/06/21
Fixed: GITHUB-1017 Reporter.log is ignored in skipped test listener (Scott Kirkpatrick)
New: Minimal code changes to allow TestNG to work for OpenJDK tests, which should be run with only the java.base module present.
Fixed: GITHUB-1047 IClassListener didn't work (Julien Herr)
Fixed: GITHUB-1049 IClassListener was called many time (Julien Herr)
Fixed: GITHUB-1045 TestNG swallows exceptions silently if @ClassRule is used (Gili Tzabari & Julien Herr)
Fixed: GITHUB-506 TestNG cannot find JUnit method names from Spock (@blackduck-joe & Julien Herr)
Fixed: GITHUB-1009 Iterator<Object[]> DataProvider: indices not working (Mark Fulton & Julien Herr)
Fixed: GITHUB-1030 Parameterized test class crashes when data provider returns empty array (Christoffer Sawicki & Julien Herr)
New: TestNG displays a warning when tests are using an empty data provider
6.9.11:
2016/03/26
Fixed: GITHUB-923 Refactored data provider's parameter values passing to a varargs or non-varargs method with @NoInjection handling (Nitin Verma)
New: GITHUB-933: Deprecate XmlTest#getTestParameters (Julien Herr)
Fixed: GITHUB-911: TestListener#onTestStart should be invoked if a suite configuration method fails (Harmin Parra Rueda & Julien Herr)
Fixed: GITHUB-793: Test suite with tests using dependency and priority has wrong behavior (Martin Hereu & Julien Herr)
Fixed: GITHUB-922: ITestResult doesn't contain name if a class has @Test (@dr29bart & Julien Herr)
Fixed: GITHUB-419: parallel mode was ignored with command line (@khospodarysko & Julien Herr)
New: GITHUB-932: Deprecate true/false parallel values. none is the new default value. (Julien Herr)
Fixed: GITHUB-960: testng-failed.xml gets generated even when there are no failures. (Krishnan Mahadevan)
Fixed: GITHUB-895: Changing status of test by setStatus of ITestResult (Raj Srivastava & Julien Herr)
Fixed: GITHUB-969: testng-failed.xml does not carry over the parameters of methods from origin suite xml(Ning Zhang)
6.9.10:
2015/12/15
Fixed: GITHUB-841: testName from @Test is now used and available from ITestResult#getName() and ITestResult#getTestName() (Julien Herr)
New: GITHUB-776: Add BeforeClass/AfterClass like on ITestListener (@vguna & Julien Herr)
Fixed: GITHUB-872: Enable end-users of TestNG to alter XmlSuite and XmlTest (Krishnan Mahadevan)
New: GITHUB-900: Support @Listeners in annotation transformer (Julien Herr)
New: GITHUB-898: Activate XML validation when possible (Julien Herr)
Fixed: GITHUB-889: XmlSuite in nested directories results in FIleNotFoundException (Virender Singh)
Fixed: GITHUB-811: Timeout is not working with parallel=tests (@michael-yxf & Julien Herr)
Fixed: GITHUB-839: Missing encoding meta data for report file (@banbq & Julien Herr)
Fixed: GITHUB-876: NullPointerException creating tests with parameters by a factory (@vixgeo & Julien Herr)
New: GITHUB-886: Add some checks on factory methods (Julien Herr)
New: GITHUB-874 / GITHUB-875 / GITHUB-882 / GITHUB-850 : Some code cleanup (Testo Nakada)
Fixed: GITHUB-866 / GITHUB-869 : Some attributes were not cloned when XmlSuite#clone was used (Virender Singh)
Fixed: GITHUB-842: Add TestResult#getTestName() support for @Test(testName) (Julien Herr)
Fixed: GITHUB-908: Fix Double.NaN assertion (Julien Herr)
6.9.9:
2015/10/27
Fixed: GITHUB-829: Allowing suites to have duplicate names. You can now configure the same suite-file to run multiple times. (Eduardo Born)
Fixed: GITHUB-834: nested suites not supported by 'testnames' (Tibor Digana & Julien Herr)
6.9.8:
2015/10/12
Replace 6.9.7 that was build with Java8 by error.
6.9.7:
No official release
Fixed: GITHUB-798: Set suitethreadpoolsize for Maven Surefire (Jan Dundáček)
Fixed: GITHUB-171: ISuiteListener methods called multiple times if multiple test elements (Daniel Qian & Julien Herr)
Fixed: GITHUB-169: IInvokedMethodListener methods executed several times before/after each test method (Mario Duarte & Julien Herr)
Fixed: GITHUB-154: MethodInterceptor will be called twice (Tim wu & Julien Herr)
6.9.6:
2015/07/15
New: GITHUB-717: Add assertThrows and expectThrows (Ryan Schmitt)
Fixed: GITHUB-755: Fixed reporting of retried tests (Ryan Schmitt)
Fixed: GITHUB-773: Test should not be skipped when the exception is expected (@CandyLiuM & Julien Herr)
6.9.5:
2015/07/12
Fixed: The ServiceLoaderTest on Windows (Mathieu Sebire)
Fixed: GITHUB-691: Fix classloading issue when using TestNG 6.9.4 and JMockit. (Mathieu Sebire)
Fixed: GITHUB-686: IAnnotationTransformer.transform is called for methods with testClass populated. (Łukasz Rekucki & Julien Herr)
Fixed: GITHUB-420: Before/AfterSuite methods may not run, when use inheritance, and enabled=false (Jakub Tokaj & Julien Herr)
Fixed: GITHUB-697: Make addFailedInvocationNumber thread-safe (Ryan Schmitt)
Fixed: GITHUB-698: Fix exit code reporting when IRetryAnalyzer is used (Ryan Schmitt)
Fixed: GITHUB-465: assertEquals(Collection, Collection) prints "null" when collections are different sizes (Michael Diamond)
New: GITHUB-710: AppVeyor is used for continuous integration on Windows (Julien Herr)
Fixed: GITHUB-599: IHookable ignored when a timeout is set (@ryanlevell & Julien Herr)
New: GITHUB-723: Allow users to add their own suite parser (Julien Herr)
Fixed: Allow '-testnames' option to work with '-xmlpathinjar' (@earthling)
Fixed: GITHUB-739: TestNG skips all test classes from suite when a @BeforeClass fails (Priyanshu Shekhar & Julien Herr)
Fixed: GITHUB-471: If @beforeMethod or @afterMethod fails then all children of the same base class will be skipped (Anton Panferov & Julien Herr)
Fixed: GITHUB-595: testng hang at switching test cases when running test cases with high thread count (vit0rg)
Eclipse:
Fixed: The 57% freeze bug (Patrick Hensley and @denyska)
6.9.4:
2015/05/09
Added: GITHUB-631: Avoid the static limitation of external DataProvider. (Julien Herr)
Added: GITHUB-631: Allow to use Guice injection in DataProvider. (Julien Herr)
Added: Drop support of Java6 and previous.
Added: GITHUB-617: Allow injection of org.testng.ITestContext into the guice parent module. (Julien Herr)
Fixed: GITHUB-606: RetryAnalyzer loops endlessly. (Krishnan Mahadevan)
Fixed: GITHUB-618: Start TestNG from jar cause recursive run of tests from packages in Suite XML without ".*" on the end (Stas Gromov)
Fixed: GITHUB-639: Typo on preserveOrder (tabei-k & Julien Herr)
Fixed: GITHUB-632: Typo in doc (Pétur Ingi Egilsson & Julien Herr)
Fixed: GITHUB-629: InvokedMethod doesn't recognize configuration method (Jan Mewes & Julien Herr)
Fixed: GITHUB-615: XmlSuite, XmlTest: Time-out tag not preserved (jphollingworth & Julien Herr)
New: GITHUB-638: Travis CI is used for continuous integration (Julien Herr)
New: GITHUB-647: SonarQube is used to follow technical debt (Julien Herr)
Added: GITHUB-616: org.testng.internal.Version will be always up-to-date (Julien Herr)
Fixed: GITHUB-634: Review of the collections package (Julien Herr)
Fixed: GITHUB-624: Fixed failure/error inversion in JUnitReportReporter (Jerome Jacob)
Fixed: GITHUB-545: TestNG running JUnit tests but not reporting all results for parameterized tests (Jonathan Leitschuh & jdillet)
Fixed: GITHUB-610: CustomizedSuites must be saved using utf-8 encoding (Juha Heljoranta)
Fixed: GITHUB-602: NoClassDefFoundError in TestNGClassFinder.<init> (aanno)
Fixed: GITHUB-529: Close InputStream and OuputStream after use (Andrew Gaul)
Fixed: GITHUB-532: Create the parent directory if it's missing (Ion Savin)
Fixed: GITHUB-541: Some OSGi manifest fixes (Evgeny Zhuravlev)
Fixed: GITHUB-657: Fix OSGI Import-Package to make jUnit4 dependency optional (Xavier Fournet)
Fixed: GITHUB-523: externally synchronize our use of the static SimpleDateFormat (mcosby)
Fixed: GITHUB-477: Typo in DTD attribute comment (Kamil Szymański)
Fixed: GITHUB-353: Typo in documentation (Jan Święcki)
Fixed: GITHUB-656: Upgrade to JCommander 1.48 (Ryan Schmitt)
Fixed: GITHUB-582: TestNG tests don't pass reliably on JDK8 (Ryan Schmitt)
New: GITHUB-645: TestNG project on Google Code redirect to GitHub
Fixed: GITHUB-310: Upgrade Guice (kronar & Julien Herr)
Fixed: GITHUB-87: @BeforeSuite/@BeforeTest methods happens to be disabled by mistake (romlom & Julien Herr)
Fixed: GITHUB-425: Wrong invocation order with lastTimeOnly (Rafael Winterhalter & Julien Herr)
Fixed: GITHUB-417: Expected Exceptions Message fails to match multi-line messages (Michael Diamond)
New: GITHUB-663: Add Guice Stage configuration for a suite (Clément Guillaume)
6.8.21:
2015/02/02
6.8.15:
2015/01/14
Fixed: OutOfMemoryException while generating reports.
Fixed: GITHUB-566: Build does not fail when successPercentage for @Test is not met
Fixed: XmlTest#setGroupInstances was not being shown in toXml().
Fixed: GITHUB-376: Some results can be lost (Konstantin Savin).
Fixed: Handle relative paths of Suite XML files properly (Nalin Makar)
6.8.5:
2013/05/13
Fixed: the OutOfMemoryException in reports
Fixed: Surefire + listeners "Can't load class" problem
===========================================================================
6.8.1
2013/03/30
Added: Descriptions in the HTML reports
Added: Various improvements to EmailableReporter (Abraham Lin)
Added: Allow injection of java.lang.reflect.Constructor and org.testng.ITestNGMethod into DataProvide (Vladislav Rassokhin)
Fixed: Assertions in the Assertions class were not failing properly.
Fixed: GITHUB-337: ConfigurationMethod#m_instance set to Boolean.FALSE due to incorrect constructor call in clone() + auto-boxing (davidely)
Fixed: Fix NPE for dependency methods/groups (Krishnan Mahadevan)
Fixed: preserve-order bug (found by VladSarrokhin).
Fixed: GITHUB-300: OutOfMemoryException from reporters when there are a lot of tests
Fixed: GITHUB-137: Main parameters with a default value should be overridden if a main parameter is specified
Fixed: GITHUB-107: Allow enum values without converting them to uppercase.
Fixed: @Guice with no modules specified is now supported
Fixed: Reporter.log() invoked from listeners were being discarded
Eclipse:
Added: Predefined listeners (Tim Wu)
Fixed: Compare dialog
===========================================================================
6.7
2012/07/15
Added: Big performance improvement when generating the reports (Frank Pavageau)
Added: <dependencies> allows you to specify group dependencies in testng.xml
Added: Blow up early if trying to include/exclude an unknown method
Added: <parameters> can now be specified under <include> (Storm Qi)
Added: GITHUB-243: Add Reporter Output per Test in XMLReporter (dunse)
Fixed: Better HTML escaping of the stack traces
Fixed: The failed assertions now use [] as delimiters instead of <> (better for the HTML reports)
Fixed: GITHUB-237: Wrong time format in XML reporter
Fixed: Threads were started sequentially instead of being interleaved
Fixed: dataProvider(parallel = true) was not killing its threads properly
Fixed: XmlSuite#toXml wasn't outputting the <groups> tag correctly
Fixed: testng-failed.xml was not carrying over the parameters from the original testng.xml
Fixed: BeforeClass failing in parent failed to skip methods in sub classes
Fixed: Better error message if <suite name=""> is missing
Fixed: GITHUB-221: Honor excludeGroups on testng tests when run in mixed mode (criccio)
Fixed: dependsOnGroups = {regexp} wasn't working properly (Alistair Ward)
Fixed: GITHUB-205: white-space was spelled whitespace in testng.css (carlin-scott)
Eclipse:
Fixed: Environment is not transferred when rerunning failed tests.
Fixed: Rerunning failed tests will preserve the environment of the original launch
===========================================================================
6.5.1
04/10/2012
Added: <suite allow-return-values="true"> (and in <test> as well)
Added: data-provider attribute to testng-results.xml
Added: Reporter display the results in the same order as test methods (Libor Zoubek)
Added: Support for running JUnit 4 tests (Lukas Jungmann)
Added: Ability to auto-detect JUnit tests ('-mixed' mode) (Lukas Jungmann)
Added: Support for ResourceCollections in an Ant tasks (requires Ant >= 1.7.0) (Lukas Jungmann)
Fixed: GITHUB-198: JUnitReportsReporter use commas in certain locales, which JUnitReports doesn't like
Fixed: GITHUB-173: Dependent methods executed out-of-order if method names match across classes (jjedMoriAnktah)
Fixed: ThreadLocal<ITestResult> leak (aslakknutsen)
Fixed: In the HTML reports, only show the first 100 characters of the parameters
Fixed: SkippedException are considered as real exception with @Test(expectedExceptions)
Eclipse:
Fixed: Java constants are properly resolved if they are used as group names (susanin)
Fixed: @Test(groups = Foo.CONSTANT) (susanin)
Fixed: Failed tests with allow-return-values="true" were not rerun
Added: <suite allow-return-values="true"> (and in <test> as well)
===========================================================================
6.4:
02/15/2012
Added: @DataProvider(indices) to return specific indices of a data provider
Added: New HTML reports
Added: configfailurepolicy=continue with DataProviders (toddq)
Added: ITestResult#getTestContext (bpedman)
Fixed: invocationCount > 1 + timeOut wasn't timing out properly
Fixed: When running TestNG programmatically, child xml suites are not run (when added using setSuiteFIles()) (Gaurav Gupta)
Fixed: GITHUB-145: Excessive test method execution (githubCast)
Fixed: GITHUB-149: reversed arguments in failAssertEqualsNoOrder().
Fixed: EmailableReporter: methods are now *really* sorted chronologically.
Eclipse:
Added: You can now add the testng.jar sources as a library (Nick Tan)
Added: Upgraded the plug-in to 3.4+ (Nick Tan)
Added: dependsOnGroups now fully supported
Fixed: @Parameters now works with both ("foo") and ({"foo"}) (davekerber)
===========================================================================
6.3.1
10/22/2011
Added: New system property: dataproviderthreadcount (Bill Ross)
Fixed: Configuration methods were reported incorrectly in listeners.
Fixed: Was creating too many listeners (Jacek Pulut)
Fixed: IAnnotationTransformer2 beforeTest/afterTest booleans were not being set
Fixed: GITHUB-92: @BeforeTest method in a super class will be called multiple time when alwayRun = true (Bubuntux)
Fixed: GITHUB-111: @AfterClass on base classes run once too many (lrivera)
Fixed: GITHUB-107: Displaying 0 tests run if a listener modifies the parameters of the suite
===========================================================================
6.3
10/17/2011
Added: "description" attribute on <include>, made available on ITestNGMethod#getDescription
Added: RemoteTestNG waits infinitely for a connection (Aleksey Kabanov)
Fixed: A method that's both a test and a factory would not invoke its data provider
Fixed: @AfterClass was not called if one of the methods was not enabled (Aleksey Kabanov)
Fixed: Groovy access bug
Fixed: The XML parser doesn't recognize parallel="instances"
Fixed: NPE when using inner classes
Fixed: GITHUB-90: @AfterClass not being run when the class contains included and not included methods
Fixed: @AfterClass not being run in some subclassing situations
Eclipse:
Fixed: Verbose levels specified in suites not respected
Fixed: Variable substitution in VM arguments is not working properly (svenhoff)
===========================================================================
6.2
08/18/2011
Added: xmlpathinjar to the TestNG ant task
Added: TestNG can now invoke package protected constructors
Added: Injectors created by the @Guice annotation are now shared at the <test> level
Added: IConfigurationListener is now a public listener, along with a new one: IConfigurationListener2
Added: When a method fails, only dependents of the same instance will be skipped
Added: parallel=instances for factory instance parallel runs
Added: @Factory(enabled)
Fixed: JUnitReports reports now report the cumulated time @{Before,After}Method+@Test for each test method
Fixed: JUnitReports reports have the name of the <test> instead of that of the first class
Fixed: Using preserve-order with a factory that creates instances of a different class causes NPE
Fixed: GITHUB-74: Bad ordering of test methods when using a @Factory constructor with dataProvider
Fixed: Changing the test result from success to failure in a listener would still count the test as a success
Fixed: ServiceLoader wasn't resolving correctly if no service loader classloader was specified
Fixed: Better ordering with mixed priorities and dependencies
Fixed: Improved detection of graph cycles in parallel runs
Fixed: @BeforeTest was invoked multiple times if a factory is used
Fixed: GITHUB-57: Allow usage of package protected constructor of test classes
Fixed: Injecting both Object[] and Method in @BeforeMethod didn't always work
Fixed: testng-results.xml now lists the results chronologically
Fixed: @Listeners specified on a base class will only be run once per listener class (dbriones)
Fixed: -groups and -excludegroups were no longer overriding testng.xml
Eclipse:
Added: Each data provider method now has a separate node entry in the TestNG view
Fixed: Nodes in error would sometimes remain green
Fixed: The TestNG context menu no longer appears where it shouldn't
===========================================================================
6.1.1
7/5/2011
Fixed: https://github.com/cbeust/testng/issues/56 testng-results.xml was reporting the instance name instead of the method name
Fixed: NPE when using preserve-order and factories.
Fixed: Depending on a skipped method would not cause a method to be skipped
===========================================================================
6.1
6/30/2011
Possible backward incompatible changes:
- Don't mutate the value returned by XmlTest#getIncludedGroups and XmlTest#getExcludedGroups.
Instead, use addIncludedGroup/addExcludedGroup.
- Failing methods that have dependees will only cause skips in the same instance. Different
test instances will not be affected
Added: Support for ServiceLoader for ITestNGListener
Added: @Factory(dataProvider / dataProviderClass) on constructors
Added: assertNotEquals() to Assert
Added: assertArrayEquals() to AssertJUnit
Added: Nested classes are now automatically added for consideration for inclusion
Added: <suite preserve-order="true"> will cause this attribute to be propagated to all <test> tags
Added: <groups> can now be specified under a <suite>
Added: Tycho compatibility (Aleksander Pohl)
Added: New <test> and <suite> flag: group-by-instances
Added: -xmlpathinjar to specify the path of testng.xml inside a test jar file
Added: ISuite#getAllMethods, to retrieve all the methods at the start of a suite.
Added: Output ITestResult attributes in xml report (nguillaumin)
Fixed: Thread safety problem in MethodInvocationHelper (Baron Roberts)
Fixed: Group dependencies were not being skipped properly.
Fixed: Dependency failures only impact the same instance
Fixed: Static classes could cause a StackOverFlowError
Fixed: IConfigurationListener was not extending ITestNGListener
Fixed: IConfigurationListener#onConfigurationFailure was never called
Fixed: TESTNG-476: <test> tags are now run in the order found in testng.xml
Fixed: Now showing failed/skipped error messages on the console for verbose >= 2
Fixed: ITestResult#getEndMillis() return 0
Fixed: TESTNG-410: Clearer error message
Fixed: TESTNG-475: @DataProvider doesn't support varargs
Fixed: Performance problems in EmailableReporter
Fixed: TESTNG-472: Better output for assertNull()
Fixed: ConcurrentModificationException when using parallel data providers.
Fixed: TESTNG-282: Problem when including+excluding packages (addicted)
Fixed: TESTNG-471: assertEquals(Map, Map) fails if a map is a subset of the other
Fixed: JUnitReporter generates an <error> tag for successful expectedExceptions tests
Fixed: ISSUE-47: Don't allow two <test>s with same name within same suite (Nalin Makar)
Fixed: If a listener implements both ISuiteListener and IInvokedMethodListener, only one of them gets invoked
Eclipse:
Added: New quick fix "Add static import org.testng.AssertJUnit.assertXXX"
Added: New workspace wide setting: excluded stack traces, to provide shorter stack traces in the view
Added: New "Clear results" icon in the tool bar
Added: When the search filter is modified, don't update the tree live if it is too big
Added: Two new @Test refactorings (pull to class level, push to method level)
Added: JUnit conversion: @Ignore
Added: JUnit conversion: assertArrayEquals()
Added: JUnit conversion: @RunWith(Parameterized.class)
Added: Support for Hamcrest failed assertions in the compare dialog
Added: JUnit conversion: suite() methods can now either be removed, commented out or left untouched
Fixed: JUnit conversion: super.setUp()/tearDown() were being removed when extending a class other than TestCase
Fixed: "Run as" menu not appearing for methods that take a generic parameter.
Fixed: The tree was incorrect if the same class is used in different <test> tags
Fixed: When creating a new Run/Debug configuration, "Launch.label" was displayed
Fixed: TESTNG-459: TestNG menu should not always be present in context menu (Mykola Nikishov)
Fixed: Performance problems in the plug-in
Fixed: Workspace-wide XML template files are not being honored.
Fixed: @BeforeClass/@AfterClass from JUnit4 are not being properly converted
Fixed: Conversions generate @Test() instead of @Test
===========================================================================
6.0
2011/03/16
Added: @Guice(moduleFactory) and IModuleFactory
Added: @Guice(module)
Added: timeOut for configuration methods
Added: -randomizesuites (Nalin Makar)
Added: IConfigurable
Fixed: @Test(priority) was not being honored in parallel mode
Fixed: @Test(timeOut) was causing threadPoolSize to be ignored
Fixed: TESTNG-468: Listeners defined in suite XML file are ignored (Michael Benz)
Fixed: TESTNG-465: Guice modules are bound individually to an injector meaning that multiple modules can't be effectively used (Danny Thomas)
Fixed: Method selectors from suites were not properly initialized (toddq)
Fixed: Throw an error when two data providers have the same name
Fixed: Better handling of classes that don't have any TestNG annotations
Fixed: XmlTest#toXml wasn't displaying the thread-count attribute
Fixed: TESTNG-438: Regression in 5.14.1: JUnit Test Execution no longer working
Fixed: TESTNG-436: Deep Map comparison for assertEquals() (Nikolay Metchev)
Fixed: Skipped tests were not always counted.
Fixed: test listeners that throw were not reporting correctly (ansgarkonermann)
Fixed: <suite junit="true"> wasn't working.
Fixed: In parallel "methods" mode, method interceptors that remove methods would cause a lock up
Fixed: EmailableReporter now sorts methods chronologically
Fixed: TESTNG-411: Throw exception on mismatch of parameter values (via DP and/or Inject) and test parameters
Fixed: IDEA-59073: exceptions that don't match don't have stack trace printed in console (Anna Kozlova)
Fixed: IDEA's plug-in was not honoring ITest (fixed in TestResultMessage)
Fixed: Methods depending on a group they belong were skipped instead of throwing a cycle exception
Fixed: TESTNG-401: ClassCastException when using a listener from Maven
Fixed: TESTNG-186: Rename IWorkerApadter to IWorkerAdapter (Tomas Pollak)
Fixed: TESTNG-415: Assert.assertEquals() for sets and maps fails with 'null' as arguments
Fixed: typo -testRunFactory
Fixed: NPE while printing results for an empty suite (Nalin Makar)
Fixed: Invoke IInvokedMethodListener.afterInvocation after fixing results for tests expecting exceptions (Nalin Makar)
Fixed: TESTNG-441: NPE in SuiteHTMLReporter#generateMethodsChronologically caused by a race condition (Slawomir Ginter)
Eclipse:
Added: Convert to YAML
Added: New global preference: JVM args
Added: Eclipse can now monitor a test-output/ directory and update the view when a new result is created
Added: Right clicking on a class/package/project now offers a menu "TestNG/Convert to TestNG"
Added: Excluded methods are now listed in the Summary tab
Added: "Description" column in the excluded methods table
Added: Dialog box when the plug-in can't contact RemoteTestNG
Added: Double clicking on an excluded method in the Summary tab will take you to its definition
Added: If you select a package before invoking the "New TestNG class" wizard, the source and package text boxes will be auto-filled
Added: When an item is selected in a tab, the same item will be selected when switching tabs
Added: A new "Summary" tab that allows the user to see a summary of the tests, sort them by time, name, etc...
Added: It's now possible "Run/Debug As" with a right click from pretty much any element that makes sense in the tree.
Added: JUnit conversion: correctly replaces assertNull and assertNotNull
Added: JUnit conversion: removes super.setUp() and super.tearDown()
Added: JUnit conversion: removes @Override
Added: JUnit conversion: replaces @Test(timeout) with @Test(timeOut) (5.14.2.4)
Added: JUnit conversion: replaces @Test(expected) with @Test(expectedExceptions) (5.14.2.4)
Added: JUnit conversion: replaces fail() with AssertJUnit.fail() (5.14.2.2)
Added: JUnit conversion: replaces Assert with AssertJUnit (5.14.2.1)
Added: The progress bar is now orange if the suite contained skipped tests and no failures
Added: Skipped test and suite icons are now orange (previously: blue)
Added: New method shortcuts: "Alt+Shift+X N", "Alt+Shift+D N" (Sven Johansson)
Added: "Create TestNG class" context menu
Added: When generating a new class, handle overridden methods by generating mangled test method names
Fixed: Green nodes could override red parent nodes back to green
Fixed: Was trying to load the classes found in the XML template file
Fixed: Stack traces of skipped tests were not showing in the Exception view
Fixed: XML files should be run in place and not copied.
Fixed: NPE when you select a passed test and click on the Compare Result icon (Mohamed Mansour)
Fixed: When the run is over, the plug-in will no longer force the focus back to the Console view
Fixed: The counter in the progress bar sometimes went over the total number of test methods (5.14.2.9)
Fixed: org.eclipse.ui.internal.ErrorViewPart cannot be cast to org.testng.eclipse.ui.TestRunnerViewPart (5.14.2.9)
Fixed: Workspace preferences now offer the "XML template" option as well as the project specific preferences (Asiel Brumfield)
Fixed: TESTNG-418: Only last suite-file in testng.xml run by Eclipse plugin
Documentation:
Added: Section on Selenium (Felipe Knorr Kuhn)
Added: Link to an article on TestNG, Mockito and Emma in the Misc section
===========================================================================
5.14.7
2011/01/27
Release for IDEA
===========================================================================
5.14.1
2010/10/2
Fixed: TESTNG-401: ClassCastException when using a listener from Maven
===========================================================================
5.14
2010/08/28
Added: test suites can now be run in parallel with -suitethreadpoolsize
Fixed: @Listeners now aggregate through base classes
Fixed: ISuite was no longer serializable
Fixed: Injection was sometimes not working properly when used with @Parameters
Fixed: TESTNG-400: afterMethod was called after onTestFailure()
Fixed: "excludedgroups" was not working on the ant task because of a typo
Fixed: ant task error if <classfileset> is used with no classes (welex91)
Fixed: TESTNG-404: threaded tests fail due to use of non-threadsafe collections (Marcus Better)
Fixed: preserve-order was not preserving class order with dependent methods
Fixed: RetryAnalyzer wasn't working properly with factories
Fixed: The ant task was no longer supporting ',' for testclass
Eclipse:
Fixed: The plug-in wasn't running Groovy tests correctly (Andrew Eisenberg)
Fixed: TESTNG-402 [Eclipse Plug-In] NPE occurred when I run twice a custom "Run configuration" on a group
===========================================================================
5.13.1
2010/08/05
Added: -methods
Added: -configfailurepolicy (Todd Quessenberry)
Added: -methodselectors (Todd Quessenberry)
Added: @NoInjection