remove a line if regex exists ONLY in 1st column of each line in a csv

I have the following comma-delimited data (spaces used for visual clarity):

 1 2014 23 2352 19.24 7.18
 1 2014 23 2353 19.78 4.65
24 2014 23 2354 20.26 25.34
 1 2014 23 2355 26.65 42.76
 1 2014 23 2356 27.34 56.356
 1 2014 23 2357 29.24 45.4
24 2014 23 2358 30.23 345.27
 1 2014 23 2359 23.23 345.21
 1 2014 24 0000 12.53 1.98
 1 2014 24 0001 7.24  12.3
24 2014 24 0002 15.68 90.8

I would like to remove ONLY every line that starts with 24 (or keep every line that starts with 1, either way).

So the desired output would be:

 1 2014 23 2352 19.24 7.18
 1 2014 23 2353 19.78 4.65
 1 2014 23 2355 26.65 42.76
 1 2014 23 2356 27.34 56.356
 1 2014 23 2357 29.24 45.4
 1 2014 23 2359 23.23 345.21
 1 2014 24 0000 12.53 1.98
 1 2014 24 0001 7.24  12.3

I have tried

sed '/24/d' (filename)

But that removes EVERY instance of 24 throughout the file, like this (not desired!):

1 2014 23 2353 19.78 4.65
1 2014 23 2355 26.65 42.76
1 2014 23 2356 27.34 56.356
1 2014 23 2359 23.23 345.21

Points for simplicity/explanation; I know this thread will be worthwhile for many others. Thank you.