| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Website | IBM | GenA11y | QualWeb | GenA11y | Wave | GenA11y | Axe-Core | GenA11y | A11yWatch | GenA11y | Existing Tools | GenA11y | Additional Violation Types | ||||||
2 | Covered | Not Covered | Covered | Not Covered | Covered | Not Covered | Covered | Not Covered | Covered | Not Covered | Covered | Not Covered | ||||||||
3 | Adp | 6 | 6 | 0 | 3 | 3 | 0 | 2 | 2 | 0 | 2 | 2 | 0 | 2 | 2 | 0 | 11 | 11 | 0 | 7 |
4 | Agoda | 3 | 3 | 0 | 4 | 4 | 0 | 4 | 4 | 0 | 3 | 3 | 0 | 3 | 3 | 0 | 6 | 6 | 0 | 9 |
5 | Caliente | 3 | 3 | 0 | 4 | 4 | 0 | 2 | 2 | 0 | 1 | 1 | 0 | 3 | 3 | 0 | 6 | 6 | 0 | 11 |
6 | Capital One | 5 | 5 | 0 | 3 | 3 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 2 | 2 | 0 | 7 | 7 | 0 | 4 |
7 | Craigslist | 3 | 3 | 0 | 5 | 5 | 0 | 2 | 2 | 0 | 2 | 2 | 0 | 1 | 1 | 0 | 8 | 8 | 0 | 11 |
8 | Cricbuzz | 6 | 6 | 0 | 3 | 1 | 2 - 1.3.1: Info and Relationships SC 1.4.3: Contrast (Minimum) | 4 | 4 | 0 | 0 | 0 | 0 | 3 | 3 | 0 | 9 | 7 | 2 | 6 |
9 | Discord | 2 | 2 | 0 | 3 | 3 | 0 | 3 | 3 | 0 | 2 | 2 | 0 | 2 | 2 | 0 | 4 | 4 | 0 | 9 |
10 | Doordash | 3 | 2 | 1 SC 1.4.3: Contrast (Minimum) | 3 | 2 | 1 SC 1.4.3: Contrast (Minimum) | 2 | 2 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 4 | 3 | 1 | 7 |
11 | Doubleclick | 3 | 2 | 1 SC 1.3.1: Info and Relationships | 3 | 3 | 0 | 3 | 3 | 0 | 2 | 2 | 0 | 2 | 2 | 0 | 6 | 5 | 1 | 7 |
12 | Ebay | 4 | 4 | 0 | 6 | 5 | 1 SC 2.4.10 Section Headings | 2 | 2 | 0 | 1 | 1 | 0 | 3 | 3 | 0 | 9 | 8 | 1 | 6 |
13 | Fragrantica | 3 | 3 | 0 | 3 | 3 | 0 | 2 | 2 | 0 | 2 | 2 | 0 | 3 | 3 | 0 | 7 | 7 | 0 | 7 |
14 | Genius | 4 | 3 | 1 SC 1.4.3: Contrast (Minimum) | 5 | 4 | 1 SC 1.4.3: Contrast (Minimum) | 4 | 3 | 1 SC 1.4.3: Contrast (Minimum) | 2 | 2 | 0 | 2 | 2 | 0 | 9 | 8 | 1 | 11 |
15 | 3 | 3 | 0 | 3 | 2 | 1 SC 2.4.1 Bypass Blocks | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 4 | 3 | 1 | 9 | |
16 | Live | 3 | 3 | 0 | 3 | 2 | 1 SC 1.4.3: Contrast (Minimum) | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 5 | 4 | 1 | 9 |
17 | Makemytrip | 4 | 4 | 0 | 2 | 2 | 0 | 2 | 2 | 0 | 1 | 1 | 0 | 2 | 2 | 0 | 5 | 5 | 0 | 10 |
18 | NIH | 3 | 2 | 1 SC 1.3.1: Info and Relationships | 3 | 2 | 1 SC 1.4.3: Contrast (Minimum) | 0 | 0 | 0 | 1 | 1 | 0 | 2 | 2 | 0 | 7 | 5 | 2 | 7 |
19 | OpenAI | 3 | 2 | 1 SC 1.3.1: Info and Relationships | 4 | 3 | 1 SC 1.4.3: Contrast (Minimum) | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 6 | 4 | 2 | 9 |
20 | Progressive | 4 | 4 | 0 | 3 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 5 | 5 | 0 | 7 |
21 | Qualtrics | 4 | 4 | 0 | 3 | 3 | 0 | 3 | 3 | 0 | 2 | 2 | 0 | 3 | 3 | 0 | 4 | 4 | 0 | 7 |
22 | Samsung | 4 | 3 | 1 SC 1.3.1: Info and Relationships | 6 | 5 | 1 SC 1.4.3: Contrast (Minimum) | 2 | 2 | 0 | 1 | 1 | 0 | 3 | 3 | 0 | 8 | 6 | 2 | 6 |
23 | ScienceDirect | 3 | 3 | 0 | 3 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 4 | 4 | 0 | 6 |
24 | Shein | 4 | 3 | 1 SC 1.3.1: Info and Relationships | 7 | 6 | 1 SC 1.4.3: Contrast (Minimum) | 6 | 6 | 0 | 5 | 5 | 0 | 5 | 5 | 0 | 11 | 9 | 2 | 7 |
25 | Stackoverflow | 5 | 5 | 0 | 6 | 6 | 0 | 4 | 4 | 0 | 3 | 3 | 0 | 3 | 3 | 0 | 10 | 10 | 0 | 9 |
26 | Steam | 7 | 7 | 0 | 6 | 5 | 1 SC 1.4.4: Resize Text (Level AA) | 4 | 4 | 0 | 3 | 3 | 0 | 3 | 3 | 0 | 10 | 9 | 1 | 11 |
27 | USPS | 5 | 5 | 0 | 6 | 5 | 1 SC 1.4.3: Contrast (Minimum) | 4 | 3 | 1 SC 1.4.3: Contrast (Minimum) | 2 | 2 | 0 | 3 | 2 | 1 SC 1.4.3: Contrast (Minimum) | 9 | 8 | 1 | 7 |
28 | Walmart | 5 | 5 | 0 | 3 | 3 | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 6 | 6 | 0 | 5 |
29 | Yahoo | 6 | 6 | 0 | 7 | 7 | 0 | 4 | 4 | 0 | 1 | 1 | 0 | 2 | 2 | 0 | 10 | 10 | 0 | 7 |
30 | Youtube | 5 | 5 | 0 | 2 | 2 | 0 | 3 | 3 | 0 | 1 | 1 | 0 | 3 | 3 | 0 | 9 | 9 | 0 | 9 |
31 | Zerodha | 6 | 6 | 0 | 4 | 4 | 0 | 4 | 4 | 0 | 2 | 2 | 0 | 4 | 4 | 0 | 10 | 10 | 0 | 9 |
32 | Zoro | 5 | 5 | 0 | 3 | 3 | 0 | 1 | 1 | 0 | 2 | 2 | 0 | 3 | 3 | 0 | 7 | 7 | 0 | 8 |
33 | Total | 124 | 117 | 7 | 119 | 106 | 13 | 70 | 68 | 2 | 44 | 44 | 0 | 66 | 65 | 1 | 216 | 198 | 18 | 237 |
GenA11y successfully detected 198 out of 216 violation types reported by existing tools, achieving a similarity ratio of 91.67%. In addition, GenA11y was able to detect 237 additional violation types beyond those found by the combination of all existing tools, averaging 8 new violation types per website.
We identified a few areas where GenA11y was less effective compared to the existing tools, particularly in detecting violations related to SC 1.3.1 Info and Relationships and SC 1.4.3 Color Contrast. These accounted for 15 missed cases by GenA11y.