Unfortunately, this is quite difficult to accomplish in 9.3.1, but in 10.0 you can use the __file__ global in script tools to determine the folder a script is running in. The same __file__ global can be used in script tools in 9.3.1, but only if the script is marked to run out of process (in the properties dialog).
The recipe you'd use is:
os.path.join(os.path.dirname(__file__), "my_sde.sde")