Commit graph

63 commits

Author SHA1 Message Date
Giteabot
d2efd2bf73
Require repo scope for PATs for private repos and basic authentication (#24362) (#24364)
Backport #24362 by @jolheiser

> The scoped token PR just checked all API routes but in fact, some web
routes like `LFS`, git `HTTP`, container, and attachments supports basic
auth. This PR added scoped token check for them.

Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-04-26 20:57:51 -04:00
Giteabot
27dbe97542
Add actions support to package auth verification (#23729) (#24028)
Backport #23729 by @yp05327

Partly fixes https://github.com/go-gitea/gitea/issues/23642

Error info:

![image](https://user-images.githubusercontent.com/18380374/227827027-4280a368-ec9e-49e0-bb93-6b496ada7cd9.png)
ActionsUser (userID -2) is used to login in to docker in action jobs.

Due to we have no permission policy settings of ActionsUser now,
ActionsUser can only access public registry by this quick fix.

Co-authored-by: yp05327 <576951401@qq.com>
2023-04-10 10:36:21 +02:00
KN4CK3R
007d181bb5
Notify on container image create (#22806)
Fixes #22791

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-18 13:36:38 +08:00
KN4CK3R
9057a008a1
Add /$count endpoints for NuGet v2 (#22855)
Fixes #22838

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-11 19:30:44 +08:00
KN4CK3R
f8c1e14a13
Use import of OCI structs (#22765)
Fixes #22758

Otherwise we would need to rewrite the structs in `oci.go`.
2023-02-06 10:07:09 +00:00
KN4CK3R
d987ac6bf1
Add Chef package registry (#22554)
This PR implements a [Chef registry](https://chef.io/) to manage
cookbooks. This package type was a bit complicated because Chef uses RSA
signed requests as authentication with the registry.


![grafik](https://user-images.githubusercontent.com/1666336/213747995-46819fd8-c3d6-45a2-afd4-a4c3c8505a4a.png)


![grafik](https://user-images.githubusercontent.com/1666336/213748145-d01c9e81-d4dd-41e3-a3cc-8241862c3166.png)

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-06 09:49:21 +08:00
KN4CK3R
df789d962b
Add Cargo package registry (#21888)
This PR implements a [Cargo registry](https://doc.rust-lang.org/cargo/)
to manage Rust packages. This package type was a little bit more
complicated because Cargo needs an additional Git repository to store
its package index.

Screenshots:

![grafik](https://user-images.githubusercontent.com/1666336/203102004-08d812ac-c066-4969-9bda-2fed818554eb.png)

![grafik](https://user-images.githubusercontent.com/1666336/203102141-d9970f14-dca6-4174-b17a-50ba1bd79087.png)

![grafik](https://user-images.githubusercontent.com/1666336/203102244-dc05743b-78b6-4d97-998e-ef76341a978f.png)

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-05 18:12:31 +08:00
KN4CK3R
6ba9ff7b48
Add Conda package registry (#22262)
This PR adds a [Conda](https://conda.io/) package registry.
2023-02-01 12:30:39 -06:00
KN4CK3R
d283a31f03
Check quota limits for container uploads (#22450)
The test coverage has revealed that container packages were not checked
against the quota limits.
2023-01-29 11:34:29 -06:00
KN4CK3R
3510d7e33a
Fix container blob mount (#22226) 2023-01-16 17:35:48 -05:00
KN4CK3R
3fef47b41c
Use ErrInvalidArgument in packages (#22268)
Related to
https://github.com/go-gitea/gitea/pull/22262#discussion_r1059010774

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-31 12:49:37 +01:00
KN4CK3R
a35749893b
Move convert package to services (#22264)
Addition to #22256

The `convert` package relies heavily on different models which is
[disallowed by our definition of
modules](https://github.com/go-gitea/gitea/blob/main/CONTRIBUTING.md#design-guideline).
This helps to prevent possible import cycles.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-29 10:57:15 +08:00
Lunny Xiao
ca67c5a8a7
refactor auth interface to return error when verify failure (#22119)
This PR changed the Auth interface signature from 
`Verify(http *http.Request, w http.ResponseWriter, store DataStore, sess
SessionStore) *user_model.User`
to 
`Verify(http *http.Request, w http.ResponseWriter, store DataStore, sess
SessionStore) (*user_model.User, error)`.

There is a new return argument `error` which means the verification
condition matched but verify process failed, we should stop the auth
process.

Before this PR, when return a `nil` user, we don't know the reason why
it returned `nil`. If the match condition is not satisfied or it
verified failure? For these two different results, we should have
different handler. If the match condition is not satisfied, we should
try next auth method and if there is no more auth method, it's an
anonymous user. If the condition matched but verify failed, the auth
process should be stop and return immediately.

This will fix #20563

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Jason Song <i@wolfogre.com>
2022-12-28 13:53:28 +08:00
KN4CK3R
86ace4b5c2
Normalize NuGet package version on upload (#22186)
Fixes #22178

After this change upload versions with different semver metadata are
treated as the same version and trigger a duplicated version error.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-20 22:20:23 -05:00
KN4CK3R
07461e18d3
Fix condition for is_internal (#22095)
depends on #22094

Fixes https://codeberg.org/forgejo/forgejo/issues/77

The old logic did not consider `is_internal`.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-12-14 17:16:01 +08:00
silverwind
0585ac3ac6
Update go dev dependencies (#22064)
`golangci-lint`
[deprecated](https://github.com/golangci/golangci-lint/issues/1841) a
bunch of linters, removed them.
2022-12-08 16:21:37 +08:00
Lunny Xiao
0a7d3ff786
refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00
Jason Song
9607750b5e
Replace fmt.Sprintf with hex.EncodeToString (#21960)
`hex.EncodeToString` has better performance than `fmt.Sprintf("%x",
[]byte)`, we should use it as much as possible.

I'm not an extreme fan of performance, so I think there are some
exceptions:

- `fmt.Sprintf("%x", func(...)[N]byte())`
- We can't slice the function return value directly, and it's not worth
adding lines.
    ```diff
    func A()[20]byte { ... }
    - a := fmt.Sprintf("%x", A())
    - a := hex.EncodeToString(A()[:]) // invalid
    + tmp := A()
    + a := hex.EncodeToString(tmp[:])
    ```
- `fmt.Sprintf("%X", []byte)`
- `strings.ToUpper(hex.EncodeToString(bytes))` has even worse
performance.
2022-11-28 11:19:18 +00:00
flynnnnnnnnnn
e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
KN4CK3R
a1ae83f36e
Workaround for container registry push/pull errors (#21862)
This PR addresses #19586

I added a mutex to the upload version creation which will prevent the
push errors when two requests try to create these database entries. I'm
not sure if this should be the final solution for this problem.

I added a workaround to allow a reupload of missing blobs. Normally a
reupload is skipped because the database knows the blob is already
present. The workaround checks if the blob exists on the file system.
This should not be needed anymore with the above fix so I marked this
code to be removed with Gitea v1.20.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-25 13:47:46 +08:00
KN4CK3R
fc7a2d5a95
Add support for HEAD requests in Maven registry (#21834)
Related #18543

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-24 16:25:13 +02:00
Jason Song
e4eaa68a2b
Replace yaml.v2 with yaml.v3 (#21832)
I don't see why we have to use two versions of yaml. The difference
between the two versions has nothing to do with our usage.
2022-11-21 16:36:59 +08:00
KN4CK3R
044c754ea5
Add context.Context to more methods (#21546)
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-19 16:12:33 +08:00
KN4CK3R
43ab9324c5
Fix setting HTTP headers after write (#21833)
The headers can't be modified after it was send to the client.
2022-11-18 01:55:15 +08:00
Lunny Xiao
34283a74e8
Allow detect whether it's in a database transaction for a context.Context (#21756)
Fix #19513

This PR introduce a new db method `InTransaction(context.Context)`,
and also builtin check on `db.TxContext` and `db.WithTx`.
There is also a new method `db.AutoTx` has been introduced but could be used by other PRs.

`WithTx` will always open a new transaction, if a transaction exist in context, return an error.
`AutoTx` will try to open a new transaction if no transaction exist in context.
That means it will always enter a transaction if there is no error.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-11-12 21:18:50 +01:00
zeripath
a0a425a13b
Add some documentation to packages (#21648)
In #21637 it was mentioned that the purpose of the API routes for the
packages is unclear. This PR adds some documentation.

Fix #21637

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2022-11-12 18:59:15 +00:00
KN4CK3R
20674dd05d
Add package registry quota limits (#21584)
Related #20471

This PR adds global quota limits for the package registry. Settings for
individual users/orgs can be added in a seperate PR using the settings
table.

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-09 14:34:27 +08:00
Wayne Starr
8c1d9885e5
Remove semver compatible flag and change pypi to an array of test cases (#21708)
This addresses #21707 and adds a second package test case for a
non-semver compatible version (this might be overkill though since you
could also edit the old package version to have an epoch in front and
see the error, this just seemed more flexible for the future).

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2022-11-08 09:41:39 +08:00
KN4CK3R
fd89c062bd
Allow local package identifiers for PyPI packages (#21690)
Fixes #21683

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-07 21:35:04 +02:00
KN4CK3R
7c11a73833
Fix package access for admins and inactive users (#21580)
I noticed an admin is not allowed to upload packages for other users
because `ctx.IsSigned` was not set.
I added a check for `user.IsActive` and `user.ProhibitLogin` too because
both was not checked. Tests enforce this now.

Co-authored-by: Lauris BH <lauris@nix.lv>
2022-10-24 22:23:25 +03:00
Wayne Starr
49a4464160
Allow for resolution of NPM registry paths that match upstream (#21568)
This PR fixes issue #21567 allowing for package tarball URLs to match
the upstream registry (and GitLab/JFrog Artifactory URLs). It uses a
regex to parse the filename (which contains the NPM version) and does a
fuzzy search to pull it out. The regex was built/expanded from
http://json.schemastore.org/package,
https://github.com/Masterminds/semver, and
https://docs.npmjs.com/cli/v6/using-npm/semver and is testable here:
https://regex101.com/r/OydBJq/5

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-10-24 21:50:22 +08:00
KN4CK3R
11d3677818
Enforce grouped NuGet search results (#21442)
Fixes #21434

Added tests to enforce this behaviour.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-17 01:18:09 +08:00
KN4CK3R
0e58201d1a
Add support for Chocolatey/NuGet v2 API (#21393)
Fixes #21294

This PR adds support for NuGet v2 API.

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-13 18:19:39 +08:00
Hubert Wawrzyńczyk
c41b30760b
Case-insensitive NuGet symbol file GUID (#21409)
NuGet symbol file lookup returned 404 on Visual Studio 2019 due to
case-sensitive api router. The api router should accept case-insensitive GUID.

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-12 14:53:56 +08:00
KN4CK3R
0e57ff7eee
Add generic set type (#21408)
This PR adds a generic set type to get rid of maps used as sets.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-12 13:18:26 +08:00
eleith
bbbf9a4b93
npm package registry support for bin (#21372)
Fix #21303

npm package.json supports binary packaging:
https://docs.npmjs.com/cli/v8/configuring-npm/package-json#bin

the npm registry documents that the binary references will be attached
to the abbreviated version object:

https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md#abbreviated-version-object

unfortunately their api documentation leaves this out:
https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md#abbreviated-version-objectdoc

which is likely to be the reason this was left out in gitea's initial
implementation

this response is critical for npm to install the binary in the `.bin`
folder so as to be included on the users default bin path, resulting in
immediate access to any binaries provided by the package
2022-10-08 13:24:44 +08:00
KN4CK3R
69fc510d6d
Add GET and DELETE endpoints for Docker blob uploads (#21367)
This PR adds support for
https://docs.docker.com/registry/spec/api/#get-blob-upload
https://docs.docker.com/registry/spec/api/#delete-blob-upload

Both are not required by the OCI spec but some clients call these
endpoints.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-07 23:30:59 +08:00
KN4CK3R
30ca91666e
Set SemverCompatible to false for Conan packages (#21275)
Fixes #21250
Related #20414

Conan packages don't have to follow SemVer.
The migration fixes the setting for all existing Conan and Generic
(#20414) packages.
2022-10-07 12:22:05 +08:00
KN4CK3R
0c8ce71188
Make NuGet service index publicly accessible (#21242)
Addition to #20734, Fixes #20717

The `/index.json` endpoint needs to be accessible even if the registry
is private. The NuGet client uses this endpoint without
authentification.

The old fix only works if the NuGet cli is used with `--source <name>`
but not with `--source <url>/index.json`.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-09-24 23:17:08 +08:00
Jack Vine
83680c97a7
NPM Package Registry search API endpoint (#20280)
Close #20098, in the NPM registry API, implemented to match what's described by https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md#get-v1search

Currently have only implemented the bare minimum to work with the [Unity Package Manager](https://docs.unity3d.com/Manual/upm-ui.html).

Co-authored-by: Jack Vine <jackv@jack-lemur-suse.cat-prometheus.ts.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-09-24 19:24:33 +08:00
KN4CK3R
8b8bdb30fb
Allow uppercase ASCII alphabet in PyPI package names (#21095)
The PyPI name regexp is too restrictive and only permits lowercase characters. This PR adjusts the regexp to add in support for uppercase characters.

Fix #21014
2022-09-07 21:18:51 +01:00
José Carlos
b8818a1c68
fix: PackageMetadataVersion deps (#21017)
Set DevDependencies, PeerDependencies & OptionalDependencies in npm package metadatas

Fix  #21013
2022-09-02 23:00:18 +01:00
KN4CK3R
41c76ad714
Add support for Vagrant packages (#20930)
* Add support for Vagrant boxes.

* Add authentication.

* Add tests.

* Add integration tests.

* Add docs.

* Add icons.

* Update routers/api/packages/api.go

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
2022-08-29 15:04:45 +08:00
zeripath
bb0ff77e46
Share HTML template renderers and create a watcher framework (#20218)
The recovery, API, Web and package frameworks all create their own HTML
Renderers. This increases the memory requirements of Gitea
unnecessarily with duplicate templates being kept in memory.

Further the reloading framework in dev mode for these involves locking
and recompiling all of the templates on each load. This will potentially
hide concurrency issues and it is inefficient.

This PR stores the templates renderer in the context and stores this
context in the NormalRoutes, it then creates a fsnotify.Watcher
framework to watch files.

The watching framework is then extended to the mailer templates which
were previously not being reloaded in dev.

Then the locales are simplified to a similar structure.

Fix #20210 
Fix #20211
Fix #20217

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-08-28 10:43:25 +01:00
KN4CK3R
dc0253b063
Replace ServeStream with ServeContent (#20903)
* Replace ServeStream with ServeContent.

* Update modules/timeutil/timestamp.go

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-08-25 12:05:21 -04:00
Lunny Xiao
1d8543e7db
Move some files into models' sub packages (#20262)
* Move some files into models' sub packages

* Move functions

* merge main branch

* Fix check

* fix check

* Fix some tests

* Fix lint

* Fix lint

* Revert lint changes

* Fix error comments

* Fix lint

Co-authored-by: 6543 <6543@obermui.de>
2022-08-25 10:31:57 +08:00
KN4CK3R
7009eb9a24
Add support for NuGet API keys (#20721)
* Add support for NuGet API key.

* lint

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-08-09 17:36:49 +03:00
KN4CK3R
920481340b
Keep download count on Container tag overwrite (#20728)
Co-authored-by: 6543 <6543@obermui.de>
2022-08-09 16:47:57 +03:00
KN4CK3R
fba20550f9
Add support for npm unpublish (#20688) 2022-08-09 15:23:43 +08:00
KN4CK3R
cc6927b2d8
Allow multiple files in generic packages (#20661)
* Allow multiple files in generic packages.
* Add deletion of a single file.
* Update docs.
* Change version check.

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-08-09 12:39:24 +08:00