The head of a leading disability charity says discrimination must end, to meet government targets on reducing incapacity benefit claimants.
Ian Charlesworth, managing director of The Shaw Trust, says prejudice persists among employers and politicians.
And he says the situation is made worse by low self-esteem, isolation and a lack of proper guidance.
The government said it was working hard to deal with the situation, but recognised more needed to be done.
The Shaw Trust is hosting a conference in London to look at ways of improving the job prospects for disabled people.
"The UK has the lowest economic activity rate for disabled people of any country in Europe," said Mr Charlesworth.
"And the UK spends five times less on employment programmes to get disabled people into work than the average European country."
The employment rate for disabled people is currently more than 30% below the government's target of 80%, the charity says.
And it says that although 75,000 disabled people find jobs every year, the numbers of those who become unemployed or economically inactive as a result of disability or health problems is 100,000.
"Of the 3.7m people who don't have work, 1.5m could and want to work," said Mr Charlesworth.
The cost of providing care and benefits for disabled people who are not working - combined with lost tax revenues - is higher than the amount spent on the NHS, according to the trust.
In order to turn the situation around, it believes a number of barriers must be removed:
- Prejudice on the part of employers, civil servants, politicians and other policy makers
- Lack of interest in the media and by the general public
- Negative attitudes among medical staff, social workers, teachers and parents
- Those looking for work need more confidence, and better access to information
- The benefits system needs to be more flexible, and tax credits and other incentives should be increased
- Transport needs to be more accessible
- People need more support to find and retain work
The Shaw Trust believes the government should take a clear lead in the delivery of employment services for disabled people.
And it says significantly more should be spent on employment programmes - it estimates that 15 times the current provision is necessary.
The Department for Work and Pensions said disabled people should not have to tolerate discrimination.
"Over the last eight years the government has supported radical and forward-looking initiatives to ensure that disabled people enjoy the right to a job," said a spokesperson. Green Paper
''Across government, we are driving forward our ambitious 20-year strategy to put disabled people at the heart of services and policy making on national and local levels.
"We are spending nearly �400m this year on employment programmes for disabled people this year.
"We recognise more needs to be done - and can be done - with the combined commitment of employers, public services, policy makers, campaign organisations and society at large.
"That is why we announced a Green Paper and legislation to be introduced in this parliamentary session.''
By Geoff Adams-Spink
BBC News website disability affairs correspondent BBC News