starlingx/root
Code Issues Proposed changes

downloader: handle non-standard DEB file names

Browse Source 
Binary package lists [1] currently support arbitrary URLs for the
packages we need to download, but there's a problem in the downloader
in that it assumes that basename of such URLs follows a standard naming
convention, ie <name>_<version>_<arch>.deb.

This fix allows us to specify any file name in such URLs. Downloader
will now save the deb file using the above convention, rregardles of how
the file name is spelled in the original source URL.

[1] https://opendev.org/starlingx/tools/src/branch/master/debian-mirror-tools/config/debian/common/base-bullseye.lst

TESTS
========================
* Clean mirrors/ directory, run downloader with and without this patch.
  Make sure it produces the same files before and after.
* Add this new package to base-bullseye.lst:
    bao 2.1.0 https://github.com/openbao/openbao/releases/download/v2.1.0/bao_2.1.0_linux_amd64.deb
* Re-run downloader (it fails due to file name discrepancy)
* Re-run downloader with this patch andm make sure the file is
  processed correctly

Story: 2010055
Task: 51735

Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I4cd31e065e371545052dd3c5ed20779445a8f004
This commit is contained in:
Davlet Panech 2025-02-21 16:03:58 -05:00
parent c3a6e4e2d5
commit 60118ca4f3
1 changed files with 5 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
build-tools/stx/downloader
View File

@ -302,11 +302,8 @@ class DebDownloader(BaseDownloader):
logger.info("Successfully created repository %s", repo)
return True
def download(self, _name, _version, url=None, retries=3):
if url != None:
# The "+" in url is converted to "%2B", so convert
# it back to "+" in save file.
dl_file = os.path.basename(url).replace("%2B", "+")
def download(self, _name, _version, dl_file, url=None, retries=3):
if url is not None:
ret = os.path.join(self.dl_dir, dl_file)
tmp_file = ".".join([ret, "tmp"])
utils.run_shell_cmd(["rm", "-rf", tmp_file], logger)
@ -350,9 +347,9 @@ class DebDownloader(BaseDownloader):
return None
package.candidate = candidate
deb_name = _name + '_' + _version
logger.info ('Downloading %s from %s', deb_name, package.candidate.uri)
logger.info ('Downloading %s from %s', dl_file, package.candidate.uri)
ret = package.candidate.fetch_binary(self.dl_dir)
assert os.path.basename(ret) == dl_file
if ret:
return ret
except Exception as e:
@ -497,7 +494,7 @@ class DebDownloader(BaseDownloader):
debnames = pname_epoch_arch.split('_')
deb_name = debnames[0]
ret = self.download(debnames[0], debnames[1], url)
ret = self.download(debnames[0], debnames[1], pname_arch, url)
if ret:
self.save_dl_file_names([os.path.basename (ret)])
deb_ver = debnames[1].split(":")[-1]