Common over-the-counter painkillers such as ibuprofen don't work for relieving back pain.
Only one in six patients treated with non-steroidal anti-inflammatory drugs, known as NSAIDs, achieve any significant reduction in pain.
The study by The George Institute for Global Health in Australia is their latest study to look at the effectiveness of existing painkillers to treat back pain.
Earlier research found paracetamol was ineffective and opioids provided minimal benefit over placebo.
Back pain is very common and normally improves within a few weeks or months with an estimated sixth of Britons suffering from it at any one time.
Yet for some it persists and sufferers are advised by the NHS to stay as active as possible, try exercises and stretches, take anti-inflammatory painkillers and use hot or cold compression packs for short-term relief.
Lead author Associate Professor Manuela Ferreira said the findings of the systematic review highlighted an urgent need to develop new therapies to treat back pain.
He said back pain will affect four fifths of Australians during their lifetime.
The Senior Research Fellow said: "Back pain is the leading cause of disability worldwide and is commonly managed by prescribing medicines such as anti-inflammatories.
"But our results show anti-inflammatory drugs actually only provide very limited short term pain relief.
"They do reduce the level of pain, but only very slightly, and arguably not of any clinical significance.
"When you factor in the side effects which are very common, it becomes clear that these drugs are not the answer to providing pain relief to the many millions of Australians who suffer from this debilitating condition every year."
The study examined 35 trials involving more than 6,000 people, also found patients taking anti-inflammatories were 2.5 times more likely to suffer from gastro-intestinal problems such as stomach ulcers and bleeding.
Research Fellow Gustavo Machado, of The George Institute and the School of Medicine at the University of Sydney, said: "Millions of Australians are taking drugs that not only don't work very well, they're causing harm. We need treatments that will actually provide substantial relief of these people's symptoms.
"Better still we need a stronger focus on preventing back pain in the first place. We know that education and exercise programs can substantially reduce the risk of developing low back pain."
Most clinical guidelines currently recommend NSAIDs as the second line analgesics after paracetamol, with opioids coming at third choice.
The study was published in the Annals of the Rheumatic Diseases.