Commit Graph

15 Commits

Author SHA1 Message Date
Mathieu Bridon
c166eadb79 distgit: Actually hardlink over the existing source at the old path
Without this, the file could exist at both the old and new path, taking
the space on the disk twice.

This forces a hardlink if the file already existed at the old path.
2015-05-29 19:07:47 +02:00
Mathieu Bridon
1de198612a distgit: And so does os.makedirs 2015-05-29 18:05:04 +02:00
Mathieu Bridon
015c24618c distgit: os.link fails if the dest already exists 2015-05-29 18:00:31 +02:00
Mathieu Bridon
2e35fa64a3 distgit: Ensure the folder exists
We can't hard link the file if the folder containing the link
destination does not exist.

Hurray for testing in staging!
2015-05-29 11:59:57 +02:00
Mathieu Bridon
e570c2f271 distgit: Upload files to both the new and old path
Currently, the CGI script is set to upload files:
- to the old path if the upload uses md5
- to the new path if the upload uses sha512

The old path is as follows:
    /%(srpmname)s/%(filename)s/%(hash)s/%(filename)s

The new path is:
    /%(srpmname)s/%(filename)s/%(hashtype)s/%(hash)s/%(filename)s

This was meant to ensure compatibility with current fedpkg which
always downloads from the old path, but will eventually download from
the new path when we move to sha512.

However, working more on this, I now think it would make for a smoother
transition if we instead always stored the files at the new path, but
just hardlinked to the old path if the upload is using md5.

This is what this patch achieves.

With this deployed in production, fedpkg could be patched to try
downloading from the new path, and fallback to the old one if necessary,
which decouples the migration to the new path from the migration to the
new hash.
2015-05-29 11:24:24 +02:00
Ralph Bean
04c59d4eee Remove the lookaside email hook (rely on fedmsg now). 2015-03-24 14:41:16 +00:00
Ralph Bean
9b5b3b531d Strip unwanted prefix from git.lookaside.new messages. 2015-02-19 15:01:14 +00:00
Ralph Bean
c50b0de2ab Try to fix lookaside.new fedmsg messages. 2015-02-19 13:26:59 +00:00
Mathieu Bridon
33ea9998d1 distgit: Add the path to the source file to the emitted messages
This is the counterpart of this change:

https://github.com/fedora-infra/fedmsg_meta_fedora_infrastructure/pull/170

Now that is has been deployed, we can start emitting the new messages.
2015-02-09 10:27:29 +01:00
Mathieu Bridon
1b2337558f lookaside: Add the hash type to the folder structure
This makes it more explicit which hash was used for a given tarball.

https://fedorahosted.org/rel-eng/ticket/5846
2014-10-28 10:59:01 +01:00
Mathieu Bridon
a92efe252a lookaside: Add support for sha512 file hashes
With this change, the upload CGI script will start preferring uploads
hashed as sha512, but still accept md5 as a fallback.

The message emitted on fedmsg is unchanged, because doing so would break
it. We're going to fix that later though.

https://fedorahosted.org/rel-eng/ticket/5846
2014-10-28 10:59:01 +01:00
Mathieu Bridon
608810be28 lookaside: De-hardcode md5 assumptions...
... as much as possible.

The point of this patch is to make it easier to move away from md5 in a
subsequent patch, without having one monster change which would be
impossible to review.

Some md5 stuff remains hardcoded, because changing it at this point
would break compatibility.

https://fedorahosted.org/rel-eng/ticket/5846
2014-10-28 10:59:01 +01:00
Mathieu Bridon
b842616187 lookaside: Drop EL < 6 support
This will make it easier to move away from md5 for the source tarballs.

It shouldn't cause any problem anyway, as Fedora runs this on EL 6 in
production, and EL 7 in staging.
2014-10-28 10:16:30 +01:00
Mathieu Bridon
76d83b13a0 lookaside: Remove tempfile when hash verification fails
Without this, tempfiles just accumulate.
2014-10-28 10:16:30 +01:00
Kevin Fenzi
278cd376ea Add dist-git-upload cgi 2014-08-26 21:19:00 +00:00