The directorate said at least 863 other pilgrims were injured in Thursday’s stampede, which took place in Mina, on the outskirts of the holy city of Mecca.
The injured have been evacuated to four different hospitals in the Mina region, according to a civil defence spokesman.
Mina is where pilgrims carry out a symbolic stoning of the devil by throwing pebbles against three stone walls. It also houses more than 160,000 tents where pilgrims spend the night during the pilgrimage.
Al Jazeera’s Basma Atassi, reporting from Mina, said the incident took place in a street between pilgrim camps.
“The street where it happened is named Street 204. This stampede did not happen at the site of the ‘stoning of the devil’ ritual, which was happening today.
“During and after the stampede the pilgrims continued to flock into Mina to perform the devil stoning ritual.”
Amateur video shared on social media showed a horrific scene, with scores of bodies – the men dressed in the simple terry cloth garments worn during Hajj – lying alongside crushed wheelchairs and water bottles.
The head of the Central Hajj Committee, Prince Khaled al-Faisal, blamed the stampede on “some pilgrims from African nationalities,” Saudi-owned al-Arabiya TV channel reported.
Iranian authorities have said that at least 43 Iranian nationals were killed in the stampede.
Survivors assessed the scene by standing on the top of roadside stalls as rescue workers in orange and yellow vests combed the area.
About 4,000 people from the rescue services were participating in the operation to help the injured and about 220 ambulances were directed to the scene, a civil defence spokesman said.
Photos released by the defence directorate on its official Twitter account showed rescue workers helping the wounded onto stretchers and loading them onto ambulances near some of the tents.
Al Jazeera’s Omar al-Saleh, reporting from Mecca, said the number of deaths could rise.
“This is only the initial number … The Hajj season was already overshadowed by the crane accident that killed 107 people and wounded more than 200,” Alsaleh said, adding: “The area has turned to a big massive construction site to allow more pilgrims to visit Mecca during Hajj.
“Mina has more than 160,000 tents divided over several camps, and with the 1.9 million people taking part in this year’s Hajj, you will understand the logistical nightmare that the Saudi authorities are facing.
“I’ve seen some footage and it’s really sad. I’ve seen piles of bodies on top of each other.”
Deadly Hajj incidents
Saudi authorities take extensive precautions to ensure the security of the Hajj and the safety of pilgrims. But tragedies are not uncommon.
In 2006, more than 360 pilgrims were killed in a stampede, also in Mina.
The day before the 2006 Hajj began, an eight-story building being used as a hostel near the Grand Mosque in Mecca collapsed, killing at least 73 people