Although very unlikely as it stands, the answer is yes, you can be prosecuted simply because your website is not accessible enough for all – not just able bodied visitors.
A disabled person has every justification to make a claim against you if your website makes it impossible or unreasonably difficult to access information and services. Especially if, when the contact you with a request to modify something, you ignore them!
If you have not made reasonable adjustments (and this means ensuring your website design is built with W3C recommendations in mind) and cannot show that this failure is justified, then you may be liable under the UK Disability Discrimination Act, and may have to pay compensation and be ordered by a court to change your site!
Fortunately, no one agency police the web looking for companies to prosecute
A useful reference is the case brought against the Sydney Olympics Committee in Australia in 2000. This legal case resulted in a landmark decision against the website owners, requiring them to pay $20,000 Australian dollars.
“This response, I am satisfied, was very hurtful for him; the suggestion that he enlist the aid of a sighted person to assist him was wholly inconsistent with his own expectations and what he himself, unaided, had been able to achieve, both at university level and in business, in spite of his disability. To dismiss him and to continue to be dismissive of him was not only hurtful, he was also made to feel, I am satisfied, various emotions including those of anger and rejection by a significant statutory agent within the community of which he himself was a part.”
Judge Hon. William Carter QC 2
It’s a very grey area and it’s been that way for years. You DO have a responsibility under UK law not to discriminate, and a disabled person has the right to take you to court over the inaccessibility of your website if you cannot justify why it’s not accessible to them!