Singularity> convertFromSource.py -i /eos/cms/store/group/phys_btag/PHM_tagger/test/Data_Zprime_bb_merged/samples.txt -o /eos/cms/store/group/phys_btag/PHM_tagger/test/elijah/etestConvert_TrainData_DF -c TrainData_DF 2022-06-02 16:50:49.356546: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2022-06-02 16:51:32,956:INFO:root: infile = /eos/cms/store/group/phys_btag/PHM_tagger/test/Data_Zprime_bb_merged/samples.txt 2022-06-02 16:51:32,956:INFO:root: outPath = /eos/cms/store/group/phys_btag/PHM_tagger/test/elijah/etestConvert_TrainData_DF 2022-06-02 16:51:32,974:INFO:root: Start conversion /afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/numpy/lib/histograms.py:1123: RuntimeWarning: invalid value encountered in true_divide hist /= s 2022-06-02 16:52:40,783:INFO:DeepJetCore.DataCollection: creating dir /dev/shm/23688 2022-06-02 16:52:40,950:INFO:root: starting /eos/cms/store/group/phys_btag/PHM_tagger/test/Data_Zprime_bb_merged/ntuple_merged_0.root... 2022-06-02 16:52:40,957:INFO:DeepJetCore.DataCollection: async started 2022-06-02 16:52:41,010:INFO:DeepJetCore.DataCollection: start cp 2022-06-02 16:52:41,056:INFO:root: starting /eos/cms/store/group/phys_btag/PHM_tagger/test/Data_Zprime_bb_merged/ntuple_merged_1.root... 2022-06-02 16:52:41,062:INFO:DeepJetCore.DataCollection: async started 2022-06-02 16:52:41,063:INFO:DeepJetCore.DataCollection: start cp 2022-06-02 16:52:41,161:INFO:root: starting /eos/cms/store/group/phys_btag/PHM_tagger/test/Data_Zprime_bb_merged/ntuple_merged_2.root... 2022-06-02 16:52:41,168:INFO:DeepJetCore.DataCollection: async started 2022-06-02 16:52:41,169:INFO:DeepJetCore.DataCollection: start cp 2022-06-02 16:52:48,089:INFO:DeepJetCore.DataCollection: convertFromSourceFile reading /dev/shm/23688/25624_tmp_ntuple_merged_0.root 2022-06-02 16:52:53,709:INFO:DeepJetCore.DataCollection: convertFromSourceFile reading /dev/shm/23688/25627_tmp_ntuple_merged_1.root 2022-06-02 16:52:55,608:INFO:DeepJetCore.DataCollection: convertFromSourceFile reading /dev/shm/23688/25630_tmp_ntuple_merged_2.root problem in ntuple_merged_0.djctd Process Process-1: 2022-06-02 16:54:12,366:INFO:root: finished /eos/cms/store/group/phys_btag/PHM_tagger/test/Data_Zprime_bb_merged/ntuple_merged_0.root... Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DataCollection.py", line 446, in writeData_async td.writeFromSourceFile(tmpinput, self.weighterobjects, istraining = not self.istestdata, outname=newpath) File "/afs/cern.ch/user/e/esherida/DeepJetCore/TrainData.py", line 108, in writeFromSourceFile self.readFromSourceFile(filename, weighterobjects, istraining) File "/afs/cern.ch/user/e/esherida/DeepJetCore/TrainData.py", line 84, in readFromSourceFile x,y,w = self.convertFromSourceFile(filename, weighterobjects, istraining, **kwargs) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DeepJet/modules/datastructures/TrainData_deepFlavour.py", line 1530, in convertFromSourceFile truth = reduceTruth(truth_arrays) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DeepJet/modules/datastructures/TrainData_deepFlavour.py", line 1494, in reduceTruth return np.vstack((b,bb+gbb,lepb,c+cc+gcc,uds,g)).transpose() File "<__array_function__ internals>", line 6, in vstack TypeError: no implementation found for 'numpy.vstack' on types that implement __array_function__: [] 2022-06-02 16:54:12,844:INFO:root: starting /eos/cms/store/group/phys_btag/PHM_tagger/test/Data_Zprime_bb_merged/ntuple_merged_3.root... 2022-06-02 16:54:12,849:INFO:root: >>>> collected result 1 of 21 2022-06-02 16:54:12,850:INFO:DeepJetCore.DataCollection: async started 2022-06-02 16:54:12,851:INFO:DeepJetCore.DataCollection: start cp Traceback (most recent call last): File "/afs/cern.ch/user/e/esherida/DeepJetCore/bin/convertFromSource.py", line 130, in checkfiles=dofilecheck File "/afs/cern.ch/user/e/esherida/DeepJetCore/DataCollection.py", line 555, in convertListOfRootFiles dir_check= not self.batch_mode File "/afs/cern.ch/user/e/esherida/DeepJetCore/DataCollection.py", line 373, in createDataFromRoot self.__writeData_async_andCollect(0, outputDir) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DataCollection.py", line 528, in __writeData_async_andCollect __collectWriteInfo(r[1][0],r[1][1],r[1][2],outputDir) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DataCollection.py", line 464, in __collectWriteInfo raise Exception("write not successful, stopping") Exception: write not successful, stopping cp: cannot create regular file '/dev/shm/23688/27018_tmp_ntuple_merged_3.root': No such file or directory Process Process-4: Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DataCollection.py", line 435, in writeData_async raise Exception("copy to ramdisk not successful for "+sample) Exception: copy to ramdisk not successful for /eos/cms/store/group/phys_btag/PHM_tagger/test/Data_Zprime_bb_merged/ntuple_merged_3.root problem in ntuple_merged_1.djctd Process Process-2: Traceback (most recent call last): File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 112, in _open self._file = numpy.memmap(self._file_path, dtype=self._dtype, mode="r") File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/numpy/core/memmap.py", line 225, in __new__ f_ctx = open(os_fspath(filename), ('r' if mode == 'c' else mode)+'b') FileNotFoundError: [Errno 2] No such file or directory: '/dev/shm/23688/25627_tmp_ntuple_merged_1.root' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 36, in __init__ self._file = open(self._file_path, "rb") FileNotFoundError: [Errno 2] No such file or directory: '/dev/shm/23688/25627_tmp_ntuple_merged_1.root' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DataCollection.py", line 446, in writeData_async td.writeFromSourceFile(tmpinput, self.weighterobjects, istraining = not self.istestdata, outname=newpath) File "/afs/cern.ch/user/e/esherida/DeepJetCore/TrainData.py", line 108, in writeFromSourceFile self.readFromSourceFile(filename, weighterobjects, istraining) File "/afs/cern.ch/user/e/esherida/DeepJetCore/TrainData.py", line 84, in readFromSourceFile x,y,w = self.convertFromSourceFile(filename, weighterobjects, istraining, **kwargs) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DeepJet/modules/datastructures/TrainData_deepFlavour.py", line 1528, in convertFromSourceFile urfile = uproot.open(filename)["deepntuplizer/tree"] File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/reading.py", line 148, in open **options, # NOTE: a comma after **options breaks Python 2 File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/reading.py", line 581, in __init__ file_path, **self._options # NOTE: a comma after **options breaks Python 2 File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 108, in __init__ self._open() File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 119, in _open self._file_path, **opts # NOTE: a comma after **opts breaks Python 2 File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 250, in __init__ self._open() File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 254, in _open [FileResource(self._file_path) for x in range(self._num_workers)] File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 254, in [FileResource(self._file_path) for x in range(self._num_workers)] File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 38, in __init__ raise uproot._util._file_not_found(file_path) FileNotFoundError: file not found '/dev/shm/23688/25627_tmp_ntuple_merged_1.root' Files may be specified as: * str/bytes: relative or absolute filesystem path or URL, without any colons other than Windows drive letter or URL schema. Examples: "rel/file.root", "C:\abs\file.root", "http://where/what.root" * str/bytes: same with an object-within-ROOT path, separated by a colon. Example: "rel/file.root:tdirectory/ttree" * pathlib.Path: always interpreted as a filesystem path or URL only (no object-within-ROOT path), regardless of whether there are any colons. Examples: Path("rel:/file.root"), Path("/abs/path:stuff.root") Functions that accept many files (uproot.iterate, etc.) also allow: * glob syntax in str/bytes and pathlib.Path. Examples: Path("rel/*.root"), "/abs/*.root:tdirectory/ttree" * dict: keys are filesystem paths, values are objects-within-ROOT paths. Example: {"/data_v1/*.root": "ttree_v1", "/data_v2/*.root": "ttree_v2"} * already-open TTree objects. * iterables of the above. problem in ntuple_merged_2.djctd Process Process-3: Traceback (most recent call last): File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 112, in _open self._file = numpy.memmap(self._file_path, dtype=self._dtype, mode="r") File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/numpy/core/memmap.py", line 225, in __new__ f_ctx = open(os_fspath(filename), ('r' if mode == 'c' else mode)+'b') FileNotFoundError: [Errno 2] No such file or directory: '/dev/shm/23688/25630_tmp_ntuple_merged_2.root' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 36, in __init__ self._file = open(self._file_path, "rb") FileNotFoundError: [Errno 2] No such file or directory: '/dev/shm/23688/25630_tmp_ntuple_merged_2.root' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DataCollection.py", line 446, in writeData_async td.writeFromSourceFile(tmpinput, self.weighterobjects, istraining = not self.istestdata, outname=newpath) File "/afs/cern.ch/user/e/esherida/DeepJetCore/TrainData.py", line 108, in writeFromSourceFile self.readFromSourceFile(filename, weighterobjects, istraining) File "/afs/cern.ch/user/e/esherida/DeepJetCore/TrainData.py", line 84, in readFromSourceFile x,y,w = self.convertFromSourceFile(filename, weighterobjects, istraining, **kwargs) File "/afs/cern.ch/user/e/esherida/DeepJetCore/DeepJet/modules/datastructures/TrainData_deepFlavour.py", line 1528, in convertFromSourceFile urfile = uproot.open(filename)["deepntuplizer/tree"] File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/reading.py", line 148, in open **options, # NOTE: a comma after **options breaks Python 2 File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/reading.py", line 581, in __init__ file_path, **self._options # NOTE: a comma after **options breaks Python 2 File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 108, in __init__ self._open() File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 119, in _open self._file_path, **opts # NOTE: a comma after **opts breaks Python 2 File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 250, in __init__ self._open() File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 254, in _open [FileResource(self._file_path) for x in range(self._num_workers)] File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 254, in [FileResource(self._file_path) for x in range(self._num_workers)] File "/afs/cern.ch/user/e/esherida/.local/lib/python3.6/site-packages/uproot/source/file.py", line 38, in __init__ raise uproot._util._file_not_found(file_path) FileNotFoundError: file not found '/dev/shm/23688/25630_tmp_ntuple_merged_2.root' Files may be specified as: * str/bytes: relative or absolute filesystem path or URL, without any colons other than Windows drive letter or URL schema. Examples: "rel/file.root", "C:\abs\file.root", "http://where/what.root" * str/bytes: same with an object-within-ROOT path, separated by a colon. Example: "rel/file.root:tdirectory/ttree" * pathlib.Path: always interpreted as a filesystem path or URL only (no object-within-ROOT path), regardless of whether there are any colons. Examples: Path("rel:/file.root"), Path("/abs/path:stuff.root") Functions that accept many files (uproot.iterate, etc.) also allow: * glob syntax in str/bytes and pathlib.Path. Examples: Path("rel/*.root"), "/abs/*.root:tdirectory/ttree" * dict: keys are filesystem paths, values are objects-within-ROOT paths. Example: {"/data_v1/*.root": "ttree_v1", "/data_v2/*.root": "ttree_v2"} * already-open TTree objects. * iterables of the above.