使用R从URL下载多个文件不起作用

2024-07-08 • 问答

我正在尝试从多个URL循环下载多个NetCDF（.nc）格式文件。但是，当我尝试打开文件时，它们似乎已损坏。

您将在下面找到我的代码。我尝试了不同的方法，例如使用download.file或system。

这是我需要下载的文件的示例： http://thredds.met.no/thredds/catalog/metusers/senorge2/seNorge2/provisional_archive/PREC1d/gridded_dataset/201601/catalog.html

但是我需要下载数百个文件，因为每个文件代表一天。

到目前为止，这是我的代码：

year = c("2016","2017")
mon = c("01","02")
day = c("01","02","03","04","05","06","07","08","09","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31")
for (y in year){
  for (m in mon){
    for (d in day){
      download.file(paste("http://thredds.met.no/thredds/fileServer/metusers/senorge2/seNorge2/provisional_archive/","PREC1d/gridded_dataset/",y,m,"/seNorge_v2_0_PREC1d_grid_",d,"_",".nc",sep=""),destfile=paste("seNorge_v2_0_PREC1d_grid_",method="curl",mode="wb")

      #try(system(paste("wget ",paste("http://thredds.met.no/thredds/fileServer/metusers/senorge2/seNorge2/provisional_archive/",#                               "PREC1d/gridded_dataset/",#           intern = TRUE,ignore.stderr = TRUE,wait=TRUE))
    }
  }
}

感谢您的帮助。

谢谢！

最好

米歇尔

尝试我的代码时，我进入了一些文件503 Service Temporarily Unavailable。在这种情况下，要重试下载，请添加--retry-on-http-error=503。 Maye还会添加--random-wait。我将method从curl更改为wget，并删除了mode="wb"，因为操作符说不用于方法'“ wget”'和'“ curl”'。希望以下解决您的问题。

year = c("2016","2017")
mon = c("01","02")
day = c("01","02","03","04","05","06","07","08","09","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31")
for (y in year){
  for (m in mon){
    for (d in day){
      download.file(paste("http://thredds.met.no/thredds/fileServer/metusers/senorge2/seNorge2/provisional_archive/","PREC1d/gridded_dataset/",y,m,"/seNorge_v2_0_PREC1d_grid_",d,"_",".nc",sep=""),destfile=paste("seNorge_v2_0_PREC1d_grid_",method="wget",extra="--random-wait --retry-on-http-error=503")
    }
  }
}

当您说文件“损坏”时，您是什么意思？您如何尝试读取nc文件？

您的代码似乎可以正常工作，我可以阅读下载的文件。您可以使用R中的raster包来读取文件。另请确保已安装ncdf4软件包。

library(raster)
r = raster('seNorge_v2_0_PREC1d_grid_20160101_20160101.nc')

使用R从URL下载多个文件不起作用

monkeymonkey666 回答：使用R从URL下载多个文件不起作用

大家都在问