There are several resources containing description of open problems in mathematical systems and control theory.
The most recent resource is the book: Unsolved problems in mathematical systems and control theory, Vincent Blondel, Alexander Megretski (Eds), Princeton University Press, 2004. That book was published by Princeton University Press in July 2004 and contains more than 60 problems (see the call for submission). The book is for sale at a price of 40 USD. A PDF electronic version of the book can also be downloaded for free from the publisher's website. Updates on the status of the problems will be posted on the PUP book website and on the editors book website. These websites are maintained by Dr Roland Hildebrand.
The book published by Princeton University Press is based in part on problems
that were presented in August 2002 at a "Workshop on open problems in system
theory" held during the Symposium MTNS
2002 at the University of Notre-Dame (USA). A booklet with descriptions
of about 40 problems presented at that symposium is available here for download
(the PDF file weights 956K): MTNS
An earlier book on open problems in Mathematical systems and control theory was published by Springer Verlag: Open Problems in Mathematical Systems and Control Theory. V. D. Blondel, E. D. Sontag, M. Vidyasagar and J. C. Willems (Eds), Springer Verlag, Heidelberg, 1999. That book contains descriptions of about 50 open problems. Many of the problems were presented at a special session of the Conference on Decision and Control in 1998. Comments on these problems and (partial) solutions to some of them are available from the book website.