Very Slow MSSQL Connection when Using IP Address

Hello,
I am having an issue and need a professional, well experienced MSSQL Server person to help resolve it.
There are 2 issues i have :
1) My initial connection / Query to MSSQL Database is usually very slow. Average about 10 to 15 seconds. But once the first connection is made, subsequent queries or connection is faster and take only about 2 to 3 seconds. This happens for either connections via Instance Name or IP address.
2) When i use Instance Name in my connection string, query is faster, less than 2 seconds. But if i use IP Address and Port for connection on same machine, connection is extremely slow. Most times i get Query time out for complex queries that i do when the Connection String contains IP Address and Port. But i do same complex query on same machine, using Database instance name directly, i never get timeout and it takes a second or two to run.
Example :
Query 1 : ConnectString = "Driver={SQL Server};Server=MSSQLSERVER5;Database=SchoolMain;Uid=Admin;Pwd=admin101;"

When i use this query, recordset is fast. Less than 2 seconds.

Query 2 : ConnectString = "Provider=SQLOLEDB.1;Persist Security Info=False;User ID=Admin;Password=admin101;Initial Catalog=SchoolMain;Data Source=192.168.1.2,1433"

When i use this query, i get Query time out expired.

Notes :
> I am using same machine. For first query, i am connecting to MSSQL database directly and i do not need internet. For second query, i plug internet to my system and use the IP address to connect to same database as with Query 1
> I have read so many things on the internet like turning off the LLMNR protocol and adding entries in the hosts file to tackle Reverse DNS. I tried them but nothing changes. Connections over IP ADDRESS still very very slow.
> I am using MSSQL Server 2008

I need help with this. I want my MSSQL queries to run super fast.