Re: Use of http proxy?

I took a stab - may not be everyone's choice, but it got me going. Add the following to ucar/nc2/dataset/HttpClientManager.java, copy class files to unpacked nc-2.2.jar and rebuild the jar: 

+++ HttpClientManager.java      2007-05-29 11:00:05.378605700 +0800
@@ -72,6 +72,12 @@
    else
_client.getParams().setParameter(HttpMethodParams.USER_AGENT, "NetcdfJava/HttpClient"); 

+    String proxyHost = System.getProperty("http.proxyHost");
+    String proxyPort = System.getProperty("http.proxyPort");
+    if ((proxyHost != null) && (proxyPort != null)) {
+ _client.getHostConfiguration().setProxy(proxyHost, Integer.parseInt(proxyPort)); 
+    }
+
    setHttpClient(_client);
  }



John Caron wrote:

Hi Nick:
DConnect2 is my (NetCDF-Java library) version of the OPenDAP Java client HTTP handler, that uses http commons library, so we should work together to make this work.
It sounds like we need to add proxy support by configuring the http commons library. I havent used an http proxy before, so I will need your help. If you have a clear idea of how to proceed, let me know. 

John

Nick Bower wrote:
Bummer. Found the real exception and the source - DODS libraries are using the commons http client, which requires explicit proxy setting at an application level (ie proxies must be a supported feature of DAP). But even when skirting DNS availability using hosts file, we go from this, 

java.net.UnknownHostException: www.marine.csiro.au
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:519)
       at java.net.Socket.connect(Socket.java:469)
       at java.net.Socket.<init>(Socket.java:366)
       at java.net.Socket.<init>(Socket.java:239)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1321) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) 
       at opendap.dap.DConnect2.openConnection(DConnect2.java:218)

to this,
C:\Program Files\IDV_2.2>jre\bin\java -Dhttp.proxyHost=proxyhost -Dhttp.proxyPor 
t=3128 -Xmx512m -jar idv.jar
25/05/2007 14:52:15 org.apache.commons.httpclient.HttpMethodDirector executeWith 
Retry
INFO: I/O exception (java.net.ConnectException) caught when processing request: 
Connection timed out: connect
25/05/2007 14:52:15 org.apache.commons.httpclient.HttpMethodDirector executeWith 
Retry
INFO: Retrying request
25/05/2007 14:53:00 org.apache.commons.httpclient.HttpMethodDirector executeWith 
Retry
INFO: I/O exception (java.net.ConnectException) caught when processing request: 
Connection timed out: connect
Looking at the source below, it would indeed seem DAP does not support proxies. This is hard to believe - it's more effective than any EULA for curtailing commercial use. I'll take my grumbles to the DAP group I guess... :( 

opendap.dap.DConnect2.java:
private void openConnection(String urlString, Command command) throws IOException, DAP2Exception, ParseException { 

   initHttpClient();
   GetMethod method = new GetMethod(urlString);
   method.setFollowRedirects(true);

   if (acceptCompress)
method.setRequestHeader(new Header("Accept-Encoding", "deflate,gzip")); 

   // enable sessions
   if (allowSessions)
     method.setRequestHeader(new Header("X-Accept-Session", "true"));

   InputStream is = null;
   try {
     // Execute the method.

     int statusCode = _httpClient.executeMethod(method);
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^





Nick Bower wrote:
IDV seems to be having trouble with our http proxy (squid). I've set the following global parameters in runIDV.bat in the usual way; 

-Dhttp.proxyHost=proxyhost -Dhttp.proxyPort=3128
however IDV fails to retrieve any data in the Data Chooser. See traceback below. Could it have something to do with IDV insisting DODS URI's begin with dods:// perhaps? What is the recommended solution for using a http proxy?
By the way, I can navigate the server in the "Display" / web browsing tab. It's only the Data Chooser that fails. 

Thanks, Nick

An error has occurred:
There was an error loading the data:
Error creating data source:dods.grid with: http://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ Grid data source failed making data set: dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ Cant open dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ or as DODS dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ 
www.marine.csiro.au
java.io.FileNotFoundException: Cant open dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ or as DODS dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ 
www.marine.csiro.au
   at ucar.nc2.dataset.NetcdfDataset.openDODS(NetcdfDataset.java:380)
   at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:316)
at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:175) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:203) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:183) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:171) 
   at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:71)
at ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:726) at ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:743) at ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:335) at ucar.unidata.data.DataManager.createDataSource(DataManager.java:1186) at ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1698) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1617) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1595) at ucar.unidata.idv.IntegratedDataViewer.handleAction(IntegratedDataViewer.java:1446) 
   at ucar.unidata.idv.DefaultIdv.handleAction(DefaultIdv.java:112)
at ucar.unidata.idv.IntegratedDataViewer.handleAction(IntegratedDataViewer.java:1378) at ucar.unidata.idv.chooser.UrlChooser.loadURLInner(UrlChooser.java:250) 
   at ucar.unidata.idv.chooser.UrlChooser.loadURL(UrlChooser.java:222)
at ucar.unidata.idv.chooser.UrlChooser.doLoadInThread(UrlChooser.java:269) 
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
   at java.lang.reflect.Method.invoke(Method.java:585)
   at ucar.unidata.util.Misc$2.run(Misc.java:919)
   at ucar.unidata.util.Misc$3.run(Misc.java:947)


And another example from your own servers...
java.io.FileNotFoundException: Cant open dods://motherlode.ucar.edu:8080/thredds/dodsC/modelsNc/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_20070524_1200.nc or as DODS dods://motherlode.ucar.edu:8080/thredds/dodsC/modelsNc/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_20070524_1200.nc 
motherlode.ucar.edu
   at ucar.nc2.dataset.NetcdfDataset.openDODS(NetcdfDataset.java:380)
   at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:316)
at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:175) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:203) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:183) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:171) 
   at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:71)
at ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:726) at ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:743) at ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:335) at ucar.unidata.data.DataManager.createDataSource(DataManager.java:1186) at ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1698) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1617) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1595) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1576) at ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:610) at ucar.unidata.idv.chooser.ThreddsHandler.processNodes(ThreddsHandler.java:842) at ucar.unidata.idv.chooser.ThreddsHandler.processNode(ThreddsHandler.java:782) 
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
   at java.lang.reflect.Method.invoke(Method.java:585)
   at ucar.unidata.util.Misc$2.run(Misc.java:921)
   at ucar.unidata.util.Misc$3.run(Misc.java:947)
  • 2007 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the idvdevelopers archives: