No you don't have to be a huge market to host but to give you a few numbers, even though you didn't respond to them the last time I posted those:
Dayton, OH: City - 166,000 Metro area - 848,000 Hosted in 2009 and 2006
Boise, ID: City - 202,000 Metro area - 585,000 Hosted in 2009 and 2005
Spokane, WA: City - 205,000 Metro area - 600,000 Hosted in 2010 and 2007
Omaha, NE: City - 454,000 Metro area - 849,000 Hosted in 2008
Little Rock, AR: City - 191,000 Metro area - 685,000 Hosted in 2008
Buffalo, NY: City - 270,000 Metro area - 1.25 Million Hosted in 2007
Lexington, KY: City - 296,000 Metro area - 453,000 Hosted in 2007
Worcester, MA: City - 182,000 Metro area - Not listed but is only 40 Miles from Boston Hosted in 2005
So from the past five years the absolute smallest city was Dayton at 166k and the smallest metro area (I think a better measure) was Lexington, KY at 453,000 but there are a few rabid college basketball fans in that area to possibly boost attendance (U of Kentucky is located there)
Fargo: City - 95,000 Metro area - 200,000
Grand Forks: City - 51,000 Metro area - 97,000
So if Fargo hosted a site it would be the smallest city and metro area by half the size compared to any city in the last 5 years.
Grand Forks would be the smallest city by almost three times as many and the metro area would be almost 4.5 times smaller than any host.
I would highly doubt that the NCAA would put one of it's premier events at a place that is considered as remote and small as either of those possible ND cities.