We are in the testing phase for a similar move. We are testing since there are other factors changing for us (i.e. physical location, virtual, etc) that we want to test with our editing workflow before update our production. Besides the official doc Jayanta Poddar mention, for what it is worth, I'll add our current experience.
We are using SQL's backup and restore to move/create the SDE database from our SQL2008 to our new SQL box (2012 or 2014?). We are not jumping to 2016 since I have seen a few threads about < 10.5.x clients maybe having issues.
The backup/restore process with keep the SDE name the same as the source SDE, but some of the user names and access needed to be tweaked (or maybe not for the restore version, but for the new?? we did this 2 weeks ago, and I was out last week). Unfortunately, that is now being done by our IT hidden from me, so I'm not 100% what they had to do.....but I have seen a couple threads in the past few weeks talking about SDE permissions that should help.
Anyway, I had them backup/restore the database as a 10.2.x SDE. Before testing the "upgrading" in place, I created a couple more SDE databases (one as 10.3.1 and eventually one as 10.5.x) and did a copy and paste. All seems to work.
The snag I ran into with authorizing the new SDE databases is locating the keycode file needed. For us, this was located on our ArcGIS Server machine c:\Program Files\ESRI\License10.2\sysgen That may save you some time.
EDIT: just saw your other question...my opinion.....move it as the current version....upgrade after.