Commit graph

14 commits

Author SHA1 Message Date
Oliver Buschjost
f61292b00d Use Minio's optimized SHA-256 2021-08-27 18:21:44 +02:00
Michael Eischer
64a43228de Prefix temporary file with object id 2021-08-12 22:17:49 +02:00
Michael Eischer
ec0766cddd Don't sync directory on Windows
Calling sync on a directory on Windows just returns "The handle is invalid"
and fails.
2021-08-12 22:14:36 +02:00
Michael Eischer
2175029c9e Sync directory to disk after upload
After a file was uploaded, also sync its containing directory to disk to
make sure that also the directory entry is persisted after a system
crash.
2021-08-12 22:08:08 +02:00
Michael Eischer
82816c67e1 Atomic upload for blobs
A upload is now first saved to a temporary file before it gets renamed
to the final filename. This ensures that incomplete uploads don't leave
broken files behind (at least not under their real filename).

In addition, removing failed uploads is no longer prone to a race
condition with a retried upload. That scenario could occur when the
first upload fails partway and the server doesn't notice that
immediately. A later retry by restic will then delete the broken upload
and upload the file again. If the server notices now that the initial
upload has failed, then it will delete the correctly uploaded file.

This has been fixed by only ever deleting the temporary file during
upload.
2021-08-12 22:06:08 +02:00
Michael Eischer
16889717c6 Add option to disable integrity check on upload 2021-08-09 15:40:50 +02:00
Michael Eischer
54adcb1fc7 Verify uploaded files
Restic uses the sha256 hash to calculate filenames based on the file
content. Check on the rest-server side that the uploaded file is intact
and reject it otherwise.
2021-08-09 15:35:13 +02:00
Alexander Neumann
ba9ee5c625 Address linter issues 2021-08-09 10:49:41 +02:00
Konrad Wojas
e957e42336 Return more useful error 2021-08-09 10:49:41 +02:00
Konrad Wojas
63c8797ba3 Fix tests, add PanicOnError option
- Helper method for internal server errors with consistent logging.
- Add PanicOnError option to panic on internal server errors. This
  makes it easier to traces where the condition was hit in testing.
2021-08-09 10:49:41 +02:00
Konrad Wojas
d4cd47e503 Minor cleanup and fixes
- Do not allow '.' as path component, because it undermines depth
  checks, and add tests
- Fix GiB reporting
- Fix metrics label
- Helper function for http errors
2021-08-09 10:49:41 +02:00
Konrad Wojas
1f593fafaf Make Server use the new repo.Handler
This contains all the glue to make Server use the new repo.Handler:

- Remove all old handlers
- Add ServeHTTP to make Server a single http.Handler
- Remove Goji routing and replace by net/http and custom routing logic

Additionally, this implements two-level backup repositories.
2021-08-09 10:49:41 +02:00
Konrad Wojas
79a8785e26 Implement repo.Handler
Refactor the old HTTP handlers to fit the purpose of the new
http.Handler:

- repo.New function to instantiate a handler for a single repo (can be done
  dynamically for every request)
- Single ServeHTTP entrypoint
- Move quota management to two methods that will be implemented later
  (stubs for now)
- Move metrics update to an external function (BlobMetricFunc type)
- Use constants and options for file modes
2021-08-09 10:49:41 +02:00
Konrad Wojas
7f14414363 Add repo package with copy of old handlers.go
Copy the old handlers.go without changes for cleaer diff of
changes later.
2021-08-09 10:49:41 +02:00