Celmina's winner came just seconds after MSU's Jerica James tied the game with at 3-point basket with 21 seconds.
The Bulldogs (3-5) led by nine at halftime and stretched that lead to as many as 13 in the early stages of the game's final half. Still, Southern Miss (4-2) found a way to rally and snap a four-game losing streak in the series between the longtime rivals.
"We turned the ball over way too many times, which resulted in 11 more shots for them than we had," MSU head coach Vic Schaefer said. "Once again, we held a team to 34 percent shooting and couldn't get the win. It's hard to lose a game the way we did tonight."
Both teams struggled offensively in the early stages of the game. The Bulldogs finally eased to a 14-9 advantage on a jumper by Jessy Ward at the midway mark of the opening half. Southern Miss scored the game's next five points for a 14-14 tie.
The Lady Eagles battled ahead at 18-17 on a pair of free throws by Jamierra Faulkner. MSU answered with back-to-back 3-point baskets by James.
MSU scored the final eight points of the half and led 33-24 at the intermission.
The Lady Eagles eventually rallied. The lead was cut to seven at 49-42. Minutes later, Southern Miss scored seven straight points to force a 52-52 tie with 5:01 left. MSU again built a 56-53 lead on two Martha Alwal free throws. However, that lead would not hold.
"Twenty offensive boards really kill you," Schaefer said. "Rebounding is an attitude. We got beat in transition too many times. We've lost two games where we didn't execute at the end on defense, and I'll take responsibility for that."
For the contest, the Bulldogs hit 22 of 56 shots from the field (39.3 percent), 5 of 10 shots from 3-point range (50.0 percent) and 10 of 12 shots from the foul line (83.3 percent). The Lady Eagles hit 23 of 67 shots from the field (34.3 percent), 1 of 9 shots from the 3-point range (11.1 percent) and 14 of 22 shots from the foul line (63.6 percent).
The teams were even at 42 in rebounding. The Bulldogs had 12 assists and 22 turnovers, while the Lady Eagles had 16 assists and 11 turnovers.
Kendra Grant led MSU with 10 points. Alwal had 10 rebounds, including eight on the defensive glass.
Southern Miss received 17 points apiece from Jerontay Clemons and Tanecka Carey. Jamierra Faulkner added 11 points and 11 assists.
MSU hosts Florida Atlantic at 7 p.m. Friday in Humphrey Coliseum.