They then hid them in the ceiling of a training room.
Investigators found software, pornography and articles about making drugs and explosives on the machines.
The discovery came after IT staff flagged unusual levels of internet activity on a contractor's account.
The PCs were found in 2015 but the case has only just been made public.
A report on the incident has been published by the Ohio Inspector General's Office and forwarded to the Ohio Ethics Commission and local officials.
It describes the series of events that led to the discovery of the computers in the ceiling of a training room at Marion Correctional Institution.
On 3 July 2015, an email alert told IT staff that the daily internet threshold for a user account had been exceeded.
But that account belonged to a contractor who was not scheduled to work on the day in question.
The mysterious user had tried to access certain file-sharing sites but had been blocked by the prison network's proxy server, which denies access to certain online content.
The user "then spent the next three hours straight trying to find sites that would circumvent the proxy and our policies," an email to IT employees noted.
After investigating the source of the network traffic, a member of the IT staff and two other inmates who were with him found a network cable leading up into the ceiling.
"When I removed the ceiling tiles I found two PCs hidden in the ceiling on two pieces of plywood," the staff member wrote in an incident report.
One of the inmates who had set up the computers later described how he had used components from other PCs that were part of a computer waste recycling programme.
He then plugged his machine into an internet connection device in the prison, according to the Inspector General's report.
"And then... bam, I'm on the network," he told investigators.
A trove of data was discovered on the machines, including records of passes being issued for inmates so they could access various parts of the prison.
Forensic analysis of the hard drives also found pornography, articles about making drugs, explosives and credit cards.
One IT employee was found to have breached inventory and crime scene protection policies.
"We will thoroughly review the reports and take any additional steps necessary to prevent these types of things from happening again," the Ohio Department of Rehabilitation and Correction said in a statement.
"It is of critical importance that we provide necessary safeguards in regards to the use of technology while still providing opportunities for offenders to participate in meaningful and rehabilitative programming."