Fix#24856
Rename "context.contextKey" to "context.WebContextKey", this context is
for web context only. But the Context itself is not renamed, otherwise
it would cause a lot of changes (if we really want to rename it, there
could be a separate PR).
The old test code doesn't really test, the "install page" gets broken
not only one time, so use new test code to make sure the "install page"
could work.
## ⚠️ Breaking
The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.
Although many legacy options still work, it's encouraged to upgrade to
the new options.
The SMTP logger is deleted because SMTP is not suitable to collect logs.
If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.
## Description
Close#12082 and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)
Just like rewriting queue #24505 : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured
log)
There is a new document (with examples): `logging-config.en-us.md`
This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.
## The old problems
The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
`log.DelLogger("console")`
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
CLI.
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.
## The new design
See `logger.go` for documents.
## Screenshot
<details>
</details>
## TODO
* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
This PR is a refactor at the beginning. And now it did 4 things.
- [x] Move renaming organizaiton and user logics into services layer and
merged as one function
- [x] Support rename a user capitalization only. For example, rename the
user from `Lunny` to `lunny`. We just need to change one table `user`
and others should not be touched.
- [x] Before this PR, some renaming were missed like `agit`
- [x] Fix bug the API reutrned from `http.StatusNoContent` to `http.StatusOK`
Replace #16455Close#21803
Mixing different Gitea contexts together causes some problems:
1. Unable to respond proper content when error occurs, eg: Web should
respond HTML while API should respond JSON
2. Unclear dependency, eg: it's unclear when Context is used in
APIContext, which fields should be initialized, which methods are
necessary.
To make things clear, this PR introduces a Base context, it only
provides basic Req/Resp/Data features.
This PR mainly moves code. There are still many legacy problems and
TODOs in code, leave unrelated changes to future PRs.
This PR
- [x] Move some functions from `issues.go` to `issue_stats.go` and
`issue_label.go`
- [x] Remove duplicated issue options `UserIssueStatsOption` to keep
only one `IssuesOptions`
This PR
- [x] Move some code from `issue.go` to `issue_search.go` and
`issue_update.go`
- [x] Use `IssuesOptions` instead of `IssueStatsOptions` becuase they
are too similiar.
- [x] Rename some functions
This PR is to allow users to specify status checks by patterns. Users
can enter patterns in the "Status Check Pattern" `textarea` to match
status checks and each line specifies a pattern. If "Status Check" is
enabled, patterns cannot be empty and user must enter at least one
pattern.
Users will no longer be able to choose status checks from the table. But
a __*`Matched`*__ mark will be added to the matched checks to help users
enter patterns.
Benefits:
- Even if no status checks have been completed, users can specify
necessary status checks in advance.
- More flexible. Users can specify a series of status checks by one
pattern.
Before:
After:
---------
Co-authored-by: silverwind <me@silverwind.io>
Ref: #24638
IMO, parallelizing might run out server resources more quickly. Gitea
shouldn't use a lot of go-routine in a web handler.
And add a comment about how many repositories there could be at most.
Co-authored-by: Yarden Shoham <git@yardenshoham.com>
`http.ServeFile/ServeContent` handles `If-xxx`, `Content-Length`,
`Range` and `Etag` correctly
After this PR, storage files (eg: avatar) could be responded with
correct Content-Length.
fix#12192 Support SSH for go get
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: mfk <mfk@hengwei.com.cn>
Co-authored-by: silverwind <me@silverwind.io>
Although some features are mixed together in this PR, this PR is not
that large, and these features are all related.
Actually there are more than 70 lines are for a toy "test queue", so
this PR is quite simple.
Major features:
1. Allow site admin to clear a queue (remove all items in a queue)
* Because there is no transaction, the "unique queue" could be corrupted
in rare cases, that's unfixable.
* eg: the item is in the "set" but not in the "list", so the item would
never be able to be pushed into the queue.
* Now site admin could simply clear the queue, then everything becomes
correct, the lost items could be re-pushed into queue by future
operations.
3. Split the "admin/monitor" to separate pages
4. Allow to download diagnosis report
* In history, there were many users reporting that Gitea queue gets
stuck, or Gitea's CPU is 100%
* With diagnosis report, maintainers could know what happens clearly
The diagnosis report sample:
[gitea-diagnosis-20230510-192913.zip](https://github.com/go-gitea/gitea/files/11441346/gitea-diagnosis-20230510-192913.zip)
, use "go tool pprof profile.dat" to view the report.
Screenshots:
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Fixes#24624
This seems to have been broken in
https://github.com/go-gitea/gitea/pull/21563
Previously, this code read
```
// Are we looking at default webhooks?
if ctx.Params(":configType") == "default-hooks" {
return &orgRepoCtx{
IsAdmin: true,
Link: path.Join(setting.AppSubURL, "/admin/hooks"),
LinkNew: path.Join(setting.AppSubURL, "/admin/default-hooks"),
NewTemplate: tplAdminHookNew,
}, nil
}
// Must be system webhooks instead
return &orgRepoCtx{
IsAdmin: true,
IsSystemWebhook: true,
Link: path.Join(setting.AppSubURL, "/admin/hooks"),
LinkNew: path.Join(setting.AppSubURL, "/admin/system-hooks"),
NewTemplate: tplAdminHookNew,
}, nil
```
but was simplified to
```
return &ownerRepoCtx{
IsAdmin: true,
IsSystemWebhook: ctx.Params(":configType") == "system-hooks",
Link: path.Join(setting.AppSubURL, "/admin/hooks"),
LinkNew: path.Join(setting.AppSubURL, "/admin/system-hooks"),
NewTemplate: tplAdminHookNew,
}, nil
```
In other words, combining the `IsSystemWebhook` check into a one-liner
and forgetting that `LinkNew` also depended on it. This meant the
rendered `<form>` always POSTed to `/admin/system-hooks`, even when you
had GETed `/admin/default-hooks/gitea/new`.
Fixes#24145
To solve the bug, I added a "computed" `TargetBehind` field to the
`Release` model, which indicates the target branch of a release.
This is particularly useful if the target branch was deleted in the
meantime (or is empty).
I also did a micro-optimization in `calReleaseNumCommitsBehind`. Instead
of checking that a branch exists and then call `GetBranchCommit`, I
immediately call `GetBranchCommit` and handle the `git.ErrNotExist`
error.
This optimization is covered by the added unit test.
#### Added
- API: Create a branch directly from commit on the create branch API
- Added `old_ref_name` parameter to allow creating a new branch from a
specific commit, tag, or branch.
- Deprecated `old_branch_name` parameter in favor of the new
`old_ref_name` parameter.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Implements displaying a README.md file present in a users ```.profile```
repository on the users profile page. If no such repository/file is
present, the user's profile page remains unchanged.
Example of user with ```.profile/README.md```
Example of user without ```.profile/README.md```
This pull request closes the feature request in #12233
Special thanks to @techknowlogick for the help in the Gitea discord!
---------
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: yp05327 <576951401@qq.com>
Co-authored-by: Yarden Shoham <git@yardenshoham.com>
The `GetAllCommits` endpoint can be pretty slow, especially in repos
with a lot of commits. The issue is that it spends a lot of time
calculating information that may not be useful/needed by the user.
The `stat` param was previously added in #21337 to address this, by
allowing the user to disable the calculating stats for each commit. But
this has two issues:
1. The name `stat` is rather misleading, because disabling `stat`
disables the Stat **and** Files. This should be separated out into two
different params, because getting a list of affected files is much less
expensive than calculating the stats
2. There's still other costly information provided that the user may not
need, such as `Verification`
This PR, adds two parameters to the endpoint, `files` and `verification`
to allow the user to explicitly disable this information when listing
commits. The default behavior is true.
1. Remove unused fields/methods in web context.
2. Make callers call target function directly instead of the light
wrapper like "IsUserRepoReaderSpecific"
3. The "issue template" code shouldn't be put in the "modules/context"
package, so move them to the service package.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
# ⚠️ Breaking
Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).
If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.
Example:
```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```
Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.
# The problem
The old queue package has some legacy problems:
* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.
It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.
# The new queue package
It keeps using old config and concept as much as possible.
* It only contains two major kinds of concepts:
* The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.
There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.
Almost ready for review.
TODO:
* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)
## Code coverage:
The "modules/context.go" is too large to maintain.
This PR splits it to separate files, eg: context_request.go,
context_response.go, context_serve.go
This PR will help:
1. The future refactoring for Gitea's web context (eg: simplify the context)
2. Introduce proper "range request" support
3. Introduce context function
This PR only moves code, doesn't change any logic.
Due to #24409 , we can now specify '--not' when getting all commits from
a repo to exclude commits from a different branch.
When I wrote that PR, I forgot to also update the code that counts the
number of commits in the repo. So now, if the --not option is used, it
may return too many commits, which can indicate that another page of
data is available when it is not.
This PR passes --not to the commands that count the number of commits in
a repo
Close#24213
Replace #23830
#### Cause
- Before, in order to making PR can get latest commit after reopening,
the `ref`(${REPO_PATH}/refs/pull/${PR_INDEX}/head) of evrey closed PR
will be updated when pushing commits to the `head branch` of the closed
PR.
#### Changes
- For closed PR , won't perform these behavior: insert`comment`, push
`notification` (UI and email), exectue
[pushToBaseRepo](7422503341/services/pull/pull.go (L409))
function and trigger `action` any more when pushing to the `head branch`
of the closed PR.
- Refresh the reference of the PR when reopening the closed PR (**even
if the head branch has been deleted before**). Make the reference of PR
consistent with the `head branch`.
This gives more "freshness" to the explore page. So it's not just the
same X users on the explore page by default, now it matches the same
sort as the repos on the explore page.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Don't remember why the previous decision that `Code` and `Release` are
non-disable units globally. Since now every unit include `Code` could be
disabled, maybe we should have a new rule that the repo should have at
least one unit. So any unit could be disabled.
Fixes#20960Fixes#7525
---------
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: yp05327 <576951401@qq.com>
Some old code doesn't respect the definition of `RequireSignInView` (the
`ignSignIn` is forgotten).
After #24413, this PR will do more fixes, ~~and rename the strange
`ignSignIn` to `optSignIn`.~~
This PR is ready for review, I think we can postpone the "ignSignIn"
renaming to another, to make this PR simple and clear.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Partially for #24457
Major changes:
1. The old `signedUserNameStringPointerKey` is quite hacky, use
`ctx.Data[SignedUser]` instead
2. Move duplicate code from `Contexter` to `CommonTemplateContextData`
3. Remove incorrect copying&pasting code `ctx.Data["Err_Password"] =
true` in API handlers
4. Use one unique `RenderPanicErrorPage` for panic error page rendering
5. Move `stripSlashesMiddleware` to be the first middleware
6. Install global panic recovery handler, it works for both `install`
and `web`
7. Make `500.tmpl` only depend minimal template functions/variables,
avoid triggering new panics
Screenshot:
<details>
</details>
## Changes
- Fixes the case where a logged in user can accept an email invitation
even if their email address does not match the address in the invitation
On the @Forgejo instance of Codeberg, we discovered that forking a repo
which is already forked now returns a 500 Internal Server Error, which
is unexpected. This is an attempt at fixing this.
The error message in the log:
~~~
2023/05/02 08:36:30 .../api/v1/repo/fork.go:147:CreateFork() [E]
[6450cb8e-113] ForkRepository: repository is already forked by user
[uname: ...., repo path: .../..., fork path: .../...]
~~~
The service that is used for forking returns a custom error message
which is not checked against.
About the order of options:
The case that the fork already exists should be more common, followed by
the case that a repo with the same name already exists for other
reasons. The case that the global repo limit is hit is probably not the
likeliest.
Co-authored-by: Otto Richter <otto@codeberg.org>
Co-authored-by: Giteabot <teabot@gitea.io>
Co-Author: @wxiaoguang
It is more convenient that user just need to enter a new branch name after he selects the branch which he want to rename.
So this PR move the function of renaming branch to the page of branches list.
This PR also restyle the button of `new branch`, `download`, `delete`....
https://user-images.githubusercontent.com/33891828/235277997-413060bb-759f-430a-b5c4-df5e40ffcd28.mov
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The old `HTMLRender` is not ideal.
1. It shouldn't be initialized multiple times, it consumes a lot of
memory and is slow.
2. It shouldn't depend on short-lived requests, the `WatchLocalChanges`
needs a long-running context.
3. It doesn't make sense to use FuncsMap slice.
HTMLRender was designed to only work for GItea's specialized 400+
templates, so it's good to make it a global shared instance.
For my specific use case, I'd like to get all commits that are on one
branch but NOT on the other branch.
For instance, I'd like to get all the commits on `Branch1` that are not
also on `master` (I.e. all commits that were made after `Branch1` was
created).
This PR adds a `not` query param that gets passed down to the `git log`
command to allow the user to exclude items from `GetAllCommits`.
See [git
documentation](https://git-scm.com/docs/git-log#Documentation/git-log.txt---not)
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: @awkwardbunny
This PR adds a Debian package registry. You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.
---------
Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
There was only one `IsRepositoryExist` function, it did: `has && isDir`
However it's not right, and it would cause 500 error when creating a new
repository if the dir exists.
Then, it was changed to `has || isDir`, it is still incorrect, it
affects the "adopt repo" logic.
To make the logic clear:
* IsRepositoryModelOrDirExist
* IsRepositoryModelExist
> The scoped token PR just checked all API routes but in fact, some web
routes like `LFS`, git `HTTP`, container, and attachments supports basic
auth. This PR added scoped token check for them.
---------
Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
If the user only belongs to one org team and the org is private,
leaving the org team will redirect to `ctx.Org.OrgLink + "/teams/"`
which is inaccessible.
So we need to check whether the user still belongs to the org.
This adds a API for getting License templates. This tries to be as close
to the [GitHub
API](https://docs.github.com/en/rest/licenses?apiVersion=2022-11-28) as
possible, but Gitea does not support all features that GitHub has. I
think they should been added, but this out f the scope of this PR. You
should merge #23006 before this PR for security reasons.
Follow #22719
### Major changes
1. `ServerError` doesn't do format, so remove the `%s`
2. Simplify `RenderBranchFeed` (slightly)
3. Remove unused `BranchFeedRSS`
4. Make `feed.RenderBranchFeed` respect `EnableFeed` config
5. Make `RepoBranchTagSelector.vue` respect `EnableFeed` setting,
otherwise there is always RSS icon
6. The `(branchURLPrefix + item.url).replace('src', 'rss')` doesn't seem
right for all cases, for example, the string `src` could appear in
`branchURLPrefix`, so we need a separate `rssURLPrefix`
7. The `<a>` in Vue menu needs `@click.stop`, otherwise the menu itself
would be triggered at the same time
8. Change `<a><button></button></a>` to `<a role=button>`
9. Use `{{PathEscapeSegments .TreePath}}` instead of `{{range $i, $v :=
.TreeNames}}/{{$v}}{{end}}`
Screenshot of changed parts:
<details>
</details>
### Other thoughts
Should we remove the RSS icon from the branch dropdown list? It seems
too complex for a list UI, and users already have the chance to get the
RSS feed URL from "branches" page.
---------
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: silverwind <me@silverwind.io>
Close#23427
Co-Author: @wxiaoguang
If a repo's release setting is enabled, the logic has't changed.
Clicking the "Tags" button will jump to `/{user}/{repo}/tags` and
`templates/repo/release/list.tmpl` template will be used.
<img
src="https://user-images.githubusercontent.com/15528715/224939362-bd8974fd-08b0-4f79-a114-3389d15847ca.png"
width="600px" />
If the release setting is disabled, clicking the "Tags" button will
still jump to `/{user}/{repo}/tags` but a new template
`templates/repo/tag/list.tmpl` will be used.
<img
src="https://user-images.githubusercontent.com/15528715/233834564-74741e49-f4e9-47c8-ac12-e306642798dc.png"
width="600px" />
Since both templates above need to render the tags list, I moved the
tags list to a shared template located in
`templates/repo/tag/table.tmpl`.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Fix#22228 adding RSS feeds for branches and files.
RSS feeds are accessed through:
* [gitea]/src/branch/{branch}.rss
* [gitea]/src/branch/{branch}/{file_name}.rss
No changes have been made to the UI to expose the feed urls for branches
and files.
This change prevents Gitea from bypassing the manual approval process
for newly registered users when OIDC is used.
- Resolves https://github.com/go-gitea/gitea/issues/23392
Signed-off-by: Gary Moon <gary@garymoon.net>
The `SHOW_FOOTER_BRANDING` came from year 2015, and it seems nobody ever
uses it. It only shows an GitHub icon which seems unrelated to Gitea, it
doesn't do what document says. So, remove it.
## ⚠️ Breaking
Users can now remove the key `[other].SHOW_FOOTER_BRANDING` from their
app.ini.
- Update all tool dependencies to latest tag
- Remove unused errcheck, it is part of golangci-lint
- Include main.go in air
- Enable wastedassign again now that it's
[generics-compatible](https://github.com/golangci/golangci-lint/pull/3689)
- Restructured lint targets to new `lint-*` namespace
The old code is unnecessarily complex, and has many misuses.
Old code "wraps" a lot, wrap wrap wrap, it's difficult to understand
which kind of handler is used.
The new code uses a general approach, we do not need to write all kinds
of handlers into the "wrapper", do not need to wrap them again and
again.
New code, there are only 2 concepts:
1. HandlerProvider: `func (h any) (handlerProvider func (next)
http.Handler)`, it can be used as middleware
2. Use HandlerProvider to get the final HandlerFunc, and use it for
`r.Get()`
And we can decouple the route package from context package (see the
TODO).
# FAQ
## Is `reflect` safe?
Yes, all handlers are checked during startup, see the `preCheckHandler`
comment. If any handler is wrong, developers could know it in the first
time.
## Does `reflect` affect performance?
No. https://github.com/go-gitea/gitea/pull/24080#discussion_r1164825901
1. This reflect code only runs for each web handler call, handler is far
more slower: 10ms-50ms
2. The reflect is pretty fast (comparing to other code): 0.000265ms
3. XORM has more reflect operations already
Close#24195
Some of the changes are taken from my another fix
f07b0de997
in #20147 (although that PR was discarded ....)
The bug is:
1. The old code doesn't handle `removedfile` event correctly
2. The old code doesn't provide attachments for type=CommentTypeReview
This PR doesn't intend to refactor the "upload" code to a perfect state
(to avoid making the review difficult), so some legacy styles are kept.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
Close#7570
1. Clearly define the wiki path behaviors, see
`services/wiki/wiki_path.go` and tests
2. Keep compatibility with old contents
3. Allow to use dashes in titles, eg: "2000-01-02 Meeting record"
4. Add a "Pages" link in the dropdown, otherwise users can't go to the
Pages page easily.
5. Add a "View original git file" link in the Pages list, even if some
file names are broken, users still have a chance to edit or remove it,
without cloning the wiki repo to local.
6. Fix 500 error when the name contains prefix spaces.
This PR also introduces the ability to support sub-directories, but it
can't be done at the moment due to there are a lot of legacy wiki data,
which use "%2F" in file names.
Co-authored-by: Giteabot <teabot@gitea.io>
The old code has a lot of technical debts, eg: `repo/wiki/view.tmpl` /
`Iterate`
This PR improves the Wiki TOC display and improves the code.
---------
Co-authored-by: delvh <dev.lh@web.de>
For 2-dot direct compare, we should use the base commit in the title and
templates, as is used elsewhere, not the common ancestor which is used
for 3-dot compare. I believe that this change should have been included
in #22949.
Close#24062
At the beginning, I just wanted to fix the warning mentioned by #24062
But, the cookie code really doesn't look good to me, so clean up them.
Complete the TODO on `SetCookie`:
> TODO: Copied from gitea.com/macaron/macaron and should be improved
after macaron removed.
The idea is to use a Layered Asset File-system (modules/assetfs/layered.go)
For example: when there are 2 layers: "custom", "builtin", when access
to asset "my/page.tmpl", the Layered Asset File-system will first try to
use "custom" assets, if not found, then use "builtin" assets.
This approach will hugely simplify a lot of code, make them testable.
Other changes:
* Simplify the AssetsHandlerFunc code
* Simplify the `gitea embedded` sub-command code
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
As promised in #23817, I have this made a PR to make Release Download
URLs predictable. It currently follows the schema
`<repo>/releases/download/<tag>/<filename>`. this already works, but it
is nowhere shown in the UI or the API. The Problem is, that it is
currently possible to have multiple files with the same name (why do we
even allow this) for a release. I had written some Code to check, if a
Release has 2 or more files with the same Name. If yes, it uses the old
`attachments/<uuid>` URlL if no it uses the new fancy URL.
I had also changed `<repo>/releases/download/<tag>/<filename>` to
directly serve the File instead of redirecting, so people who who use
automatic update checker don't end up with the `attachments/<uuid>` URL.
Fixes#10919
---------
Co-authored-by: a1012112796 <1012112796@qq.com>
The first step of #23328
This PR cleans:
1. Dead function like `SubStr`
2. Unnecessary function like `UseHTTPS`, it should be provided by
`window.origin`
3. Duplicate function like `IsShowFullName`, there was already a
`DeafultShowFullName`
I have searched these removed functions globally, no use in code.
Right now the authors search dropdown might take a long time to load if
amount of authors is huge.
Example: (In the video below, there are about 10000 authors, and it
takes about 10 seconds to open the author dropdown)
https://user-images.githubusercontent.com/17645053/229422229-98aa9656-3439-4f8c-9f4e-83bd8e2a2557.mov
Possible improvements can be made, which will take 2 steps (Thanks to
@wolfogre for advice):
Step 1：
Backend: Add a new api, which returns a limit of 30 posters with matched
prefix.
Frontend: Change the search behavior from frontend search(fomantic
search) to backend search(when input is changed, send a request to get
authors matching the current search prefix)
Step 2：
Backend: Optimize the api in step 1 using indexer to support fuzzy
search.
This PR is implements the first step. The main changes:
1. Added api: `GET /{type:issues|pulls}/posters` , which return a limit
of 30 users with matched prefix (prefix sent as query). If
`DEFAULT_SHOW_FULL_NAME` in `custom/conf/app.ini` is set to true, will
also include fullnames fuzzy search.
2. Added a tooltip saying "Shows a maximum of 30 users" to the author
search dropdown
3. Change the search behavior from frontend search to backend search
After:
https://user-images.githubusercontent.com/17645053/229430960-f88fafd8-fd5d-4f84-9df2-2677539d5d08.mov
Fixes: https://github.com/go-gitea/gitea/issues/22586
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
The _graceful_ should fail less when the `.editorconfig` file isn't
properly written, e.g. boolean values from YAML or unparseable numbers
(when a number is expected). As is... information is lost as the
_warning_ (a go-multierror.Error) is ignored. If anybody knows how to
send them to the UI as warning; any help is appreciated.
Closes#20694
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
Fork a repo, and input an invalid repo name, you will get an error.
Then rename the repo name to a valid one, the submit button is still
disabled.
In other pages, for example, in create repo page, the submit button will
not be disabled.
So I removed the codes to disable the submit button.
The old code just parses an invalid key to `TypeInvalid` and uses it as
normal, and duplicate keys will be kept.
So this PR will ignore invalid key and log warning and also deduplicate
valid units.
Changes:
- `RenderWithErr` should render `tplUserEdit` not `tplUserNew` in edit
page
- If error occurred in `HandleUsernameChange` redirect to original edit
page instead of user list page
Follow #21962
After I eat my own dogfood, I would say that
ONLY_SHOW_RELEVANT_REPOS=false is necessary for many private/enterprise
instances, because many private repositories do not have
"description/topic", users just want to search by their names.
This PR also adds `PageIsExploreRepositories` check, to make code more
strict, because the `search` template is shared for different purpose.
And during the test, I found a bug that the "Search" button didn't
respect the "relevant" parameter, so this PR fixes the bug by the way
together.
I think this PR needs to be backported.
# Why this PR comes
At first, I'd like to help users like #23636 (there are a lot)
The unclear "Internal Server Error" is quite anonying, scare users,
frustrate contributors, nobody knows what happens.
So, it's always good to provide meaningful messages to end users (of
course, do not leak sensitive information).
When I started working on the "response message to end users", I found
that the related code has a lot of technical debt. A lot of copy&paste
code, unclear fields and usages.
So I think it's good to make everything clear.
# Tech Backgrounds
Gitea has many sub-commands, some are used by admins, some are used by
SSH servers or Git Hooks. Many sub-commands use "internal API" to
communicate with Gitea web server.
Before, Gitea server always use `StatusCode + Json "err" field` to
return messages.
* The CLI sub-commands: they expect to show all error related messages
to site admin
* The Serv/Hook sub-commands (for git clients): they could only show
safe messages to end users, the error log could only be recorded by
"SSHLog" to Gitea web server.
In the old design, it assumes that:
* If the StatusCode is 500 (in some functions), then the "err" field is
error log, shouldn't be exposed to git client.
* If the StatusCode is 40x, then the "err" field could be exposed. And
some functions always read the "err" no matter what the StatusCode is.
The old code is not strict, and it's difficult to distinguish the
messages clearly and then output them correctly.
# This PR
To help to remove duplicate code and make everything clear, this PR
introduces `ResponseExtra` and `requestJSONResp`.
* `ResponseExtra` is a struct which contains "extra" information of a
internal API response, including StatusCode, UserMsg, Error
* `requestJSONResp` is a generic function which can be used for all
cases to help to simplify the calls.
* Remove all `map["err"]`, always use `private.Response{Err}` to
construct error messages.
* User messages and error messages are separated clearly, the `fail` and
`handleCliResponseExtra` will output correct messages.
* Replace all `Internal Server Error` messages with meaningful (still
safe) messages.
This PR saves more than 300 lines, while makes the git client messages
more clear.
Many gitea-serv/git-hook related essential functions are covered by
tests.
---------
Co-authored-by: delvh <dev.lh@web.de>
Closes#20955
This PR adds the possibility to disable blank Issues, when the Repo has
templates. This can be done by creating the file
`.gitea/issue_config.yaml` with the content `blank_issues_enabled` in
the Repo.
Adds API endpoints to manage issue/PR dependencies
* `GET /repos/{owner}/{repo}/issues/{index}/blocks` List issues that are
blocked by this issue
* `POST /repos/{owner}/{repo}/issues/{index}/blocks` Block the issue
given in the body by the issue in path
* `DELETE /repos/{owner}/{repo}/issues/{index}/blocks` Unblock the issue
given in the body by the issue in path
* `GET /repos/{owner}/{repo}/issues/{index}/dependencies` List an
issue's dependencies
* `POST /repos/{owner}/{repo}/issues/{index}/dependencies` Create a new
issue dependencies
* `DELETE /repos/{owner}/{repo}/issues/{index}/dependencies` Remove an
issue dependency
Closes https://github.com/go-gitea/gitea/issues/15393Closes#22115
Co-authored-by: Andrew Thornton <art27@cantab.net>
Always respect the `setting.UI.ShowUserEmail` and `KeepEmailPrivate`
setting.
* It doesn't make sense to show user's own E-mail to themself.
* Always hide the E-mail if KeepEmailPrivate=true, then the user could
know how their profile page looks like for others.
* Revert the `setting.UI.ShowUserEmail` change from #4981 . This setting
is used to control the E-mail display, not only for the user list page.
ps: the incorrect `<div .../>` tag on the profile page has been fixed by
#23748 together, so this PR becomes simpler.
Remove `[repository.editor] PREVIEWABLE_FILE_MODES` setting that seemed
like it was intended to support this but did not work. Instead, whenever
viewing a file shows a preview, also have a Preview tab in the file
editor.
Add new `/markup` web and API endpoints with `comment`, `gfm`,
`markdown` and new `file` mode that uses a file path to determine the
renderer.
Remove `/markdown` web endpoint but keep the API for backwards and
GitHub compatibility.
## ⚠️ BREAKING ⚠️
The `[repository.editor] PREVIEWABLE_FILE_MODES` setting was removed.
This setting served no practical purpose and was not working correctly.
Instead a preview tab is always shown in the file editor when supported.
---------
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Since #23493 has conflicts with latest commits, this PR is my proposal
for fixing #23371
Details are in the comments
And refactor the `modules/options` module, to make it always use
"filepath" to access local files.
Benefits:
* No need to do `util.CleanPath(strings.ReplaceAll(p, "\\", "/"))),
"/")` any more (not only one before)
* The function behaviors are clearly defined
The `q` parameter was not rendered in pagination links because
`context.Pagination:AddParam` checks for existance of the parameter in
`ctx.Data` where it was absent. Added the parameter there to fix it.
---
Hi, very naive and **untested** first time ever Go code, feel free to
reject/edit this as needed.
(PyPI actually performs "naive" string comparison too)
---------
Co-authored-by: Lauris BH <lauris@nix.lv>
When attempting to migrate a repository via the API endpoint comments
are always included. This can create a problem if your source repository
has issues or pull requests but you do not want to import them into
Gitea that displays as something like:
> Error 500: We were unable to perform the request due to server-side
problems. 'comment references non existent IssueIndex 4
There are only two ways to resolve this:
1. Migrate using the web interface
2. Migrate using the API including at issues or pull requests.
This PR matches the behavior of the API migration router to the web
migration router.
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Close#23444
Add `Repository` to npm package `Metadata` struct so the `repository` in
`package.json` can be stored and be returned in the endpoint.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
A part of https://github.com/go-gitea/gitea/pull/22865
At first, I think we do not need 3 ProjectTypes, as we can check user
type, but it seems that it is not database friendly.
---------
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
This PR fixes the tags sort issue mentioned in #23432
The tags on dropdown shoud be sorted in descending order of time but are
not. Because when getting tags, it execeutes `git tag sort
--sort=-taggerdate`. Git supports two types of tags: lightweight and
annotated, and `git tag sort --sort=-taggerdate` dosen't work with
lightweight tags, which will not give correct result. This PR add
`GetTagNamesByRepoID ` to get tags from the database so the tags are
sorted.
Also adapt this change to the droplist when comparing branches.
Dropdown places:
<img width="369" alt="截屏2023-03-15 14 25 39"
src="https://user-images.githubusercontent.com/17645053/225224506-65a72e50-4c11-41d7-8187-a7e9c7dab2cb.png">
<img width="675" alt="截屏2023-03-15 14 25 27"
src="https://user-images.githubusercontent.com/17645053/225224526-65ce8008-340c-43f6-aa65-b6bd9e1a1bf1.png">
`namedBlob` turned out to be a poor imitation of a `TreeEntry`. Using
the latter directly shortens this code.
This partially undoes https://github.com/go-gitea/gitea/pull/23152/,
which I found a merge conflict with, and also expands the test it added
to cover the subtle README-in-a-subfolder case.
this is a simple endpoint that adds the ability to rename users to the
admin API.
Note: this is not in a mergeable state. It would be better if this was
handled by a PATCH/POST to the /api/v1/admin/users/{username} endpoint
and the username is modified.
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Close#23411
Always pass "page" query parameter to backend, and make backend respect
it.
The `ctx.FormInt("limit")` is never used, so removed.
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Close#22934
In `/user/repos` API (and other APIs related to creating repos), user
can specify a readme template for auto init. At present, if the
specified template does not exist, a `500` will be returned . This PR
improved the logic and will return a `400` instead of `500`.
When creating attachments (issue, release, repo) the file size (being
part of the multipart file header) is passed through the chain of
creating an attachment to ensure the MinIO client can stream the file
directly instead of having to read it to memory completely at first.
Fixes#23393
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Fixes https://github.com/go-gitea/gitea/issues/22676
Context Data `IsOrganizationMember` and `IsOrganizationOwner` is used to
control the visibility of `people` and `team` tab.
2871ea0809/templates/org/menu.tmpl (L19-L40)
And because of the reuse of user projects page, User Context is changed
to Organization Context. But the value of `IsOrganizationMember` and
`IsOrganizationOwner` are not being given.
I reused func `HandleOrgAssignment` to add them to the ctx, but may have
some unnecessary variables, idk whether it is ok.
I found there is a missing `PageIsViewProjects` at create project page.
When there is an error creating a new openIDConnect authentication
source try to handle the error a little better.
Close#23283
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
A part of https://github.com/go-gitea/gitea/pull/22865
We have edit buttons in projects list page and project view page.
But after user edit a project, it will always redirect to the projects
list page.
Replace #23350.
Refactor `setting.Database.UseMySQL` to
`setting.Database.Type.IsMySQL()`.
To avoid mismatching between `Type` and `UseXXX`.
This refactor can fix the bug mentioned in #23350, so it should be
backported.
Close#23241
Before: press Ctrl+Enter in the Code Review Form, a single comment will
be added.
After: press Ctrl+Enter in the Code Review Form, start the review with
pending comments.
The old name `is_review` is not clear, so the new code use
`pending_review` as the new name.
Co-authored-by: delvh <leon@kske.dev>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
When creating commit status for Actons jobs, a payload with nil
`HeadCommit` will cause panic.
Reported at:
https://gitea.com/gitea/act_runner/issues/28#issuecomment-732166
Although the `HeadCommit` probably can not be nil after #23215,
`CreateCommitStatus` should protect itself, to avoid being broken in the
future.
In addition, it's enough to print error log instead of returning err
when `CreateCommitStatus` failed.
---------
Co-authored-by: delvh <dev.lh@web.de>
## TLDR
* Fix the broken page / broken image problem when click "Install"
* Close#20089
* Fix the Password Hash Algorithm display problem for #22942
* Close#23183
* Close#23184
## Details
### The broken page / broken image problem when click "Install"
(Redirect failed after install gitea #23184)
Before: when click "install", all new requests will fail, because the
server has been restarted. Users just see a broken page with broken
images, sometimes the server is not ready but the user would have been
redirect to "/user/login" page, then the users see a new broken page
(connection refused or something wrong ...)
After: only check InstallLock=true for necessary handlers, and sleep for
a while before restarting the server, then the browser has enough time
to load the "post-install" page. And there is a script to check whether
"/user/login" is ready, the user will only be redirected to the login
page when the server is ready.
### During new instance setup make 'Gitea Base URL' filled from
window.location.origin #20089
If the "app_url" input contains `localhost` (the default value from
config), use current window's location href as the `app_url` (aka
ROOT_URL)
### Fix the Password Hash Algorithm display problem for "Provide the
ability to set password hash algorithm parameters #22942"
Before: the UI shows `pbkdf2$50000$50`
<details>
</details>
After: the UI shows `pbkdf2`
<details>
</details>
### GET data: net::ERR_INVALID_URL #23183
Cause by empty `data:` in `<link rel="manifest"
href="data:{{.ManifestData}}">`
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
`renderReadmeFile` needs `readmeTreelink` as parameter but gets
`treeLink`.
The values of them look like as following:
`treeLink`: `/{OwnerName}/{RepoName}/src/branch/{BranchName}`
`readmeTreelink`:
`/{OwnerName}/{RepoName}/src/branch/{BranchName}/{ReadmeFileName}`
`path.Dir` in
8540fc45b1/routers/web/repo/view.go (L316)
should convert `readmeTreelink` into
`/{OwnerName}/{RepoName}/src/branch/{BranchName}` instead of the current
`/{OwnerName}/{RepoName}/src/branch`.
Fixes#23151
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Before, the `dict "ctx" ...` map is used to pass data between templates.
Now, more and more templates need to use real Go context:
* #22962
* #23092
`ctx` is a Go concept for `Context`, misusing it may cause problems, and
it makes it difficult to review or refactor.
This PR contains 2 major changes:
* In the top scope of a template, the `$` is the same as the `.`, so the
old labels_sidebar's `root` is the `ctx`. So this `ctx` could just be
removed.
bd7f218dce
* Rename all other `ctx` to `ctxData`, and it perfectly matches how it
comes from backend: `"ctxData": ctx.Data`.
7c01260e1d
From now on, there is no `ctx` in templates. There are only:
* `ctxData` for passing data
* `Context` for Go context
Extract from #11669 and enhancement to #22585 to support exclusive
scoped labels in label templates
* Move label template functionality to label module
* Fix handling of color codes
* Add Advanced label template
The name of the job or step comes from the workflow file, while the name
of the runner comes from its registration. If the strings used for these
names are too long, they could cause db issues.
GetActiveStopwatch & HasUserStopwatch is a hot piece of code that is
repeatedly called and on examination of the cpu profile for TestGit it
represents 0.44 seconds of CPU time. This PR reduces this time to 80ms.
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <leon@kske.dev>
This includes pull requests that you approved, requested changes or
commented on. Currently such pull requests are not visible in any of the
filters on /pulls, while they may need further action like merging, or
prodding the author or reviewers.
Especially when working with a large team on a repository it's helpful
to get a full overview of pull requests that may need your attention,
without having to sift through the complete list.
Partially fix#23050
After #22294 merged, it always has a warning log like `cannot get
context cache` when starting up. This should not affect any real life
but it's annoying. This PR will fix the problem. That means when
starting up, getting the system settings will not try from the cache but
will read from the database directly.
---------
Co-authored-by: Lauris BH <lauris@nix.lv>
minio/sha256-simd provides additional acceleration for SHA256 using
AVX512, SHA Extensions for x86 and ARM64 for ARM.
It provides a drop-in replacement for crypto/sha256 and if the
extensions are not available it falls back to standard crypto/sha256.
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
The API to create tokens is missing the ability to set the required
scopes for tokens, and to show them on the API and on the UI.
This PR adds this functionality.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Close: #22910
---
I'm confused about that why does the api (`GET
/repos/{owner}/{repo}/pulls/{index}/files`) require caller to pass the
parameters `limit` and `page`.
In my case, the caller only needs to pass a `skip-to` to paging. This is
consistent with the api `GET /{owner}/{repo}/pulls/{index}/files`
So, I deleted the code related to `listOptions`
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Some bugs caused by less unit tests in fundamental packages. This PR
refactor `setting` package so that create a unit test will be easier
than before.
- All `LoadFromXXX` files has been splited as two functions, one is
`InitProviderFromXXX` and `LoadCommonSettings`. The first functions will
only include the code to create or new a ini file. The second function
will load common settings.
- It also renames all functions in setting from `newXXXService` to
`loadXXXSetting` or `loadXXXFrom` to make the function name less
confusing.
- Move `XORMLog` to `SQLLog` because it's a better name for that.
Maybe we should finally move these `loadXXXSetting` into the `XXXInit`
function? Any idea?
---------
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
This PR refactors and improves the password hashing code within gitea
and makes it possible for server administrators to set the password
hashing parameters
In addition it takes the opportunity to adjust the settings for `pbkdf2`
in order to make the hashing a little stronger.
The majority of this work was inspired by PR #14751 and I would like to
thank @boppy for their work on this.
Thanks to @gusted for the suggestion to adjust the `pbkdf2` hashing
parameters.
Close#14751
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Add a new "exclusive" option per label. This makes it so that when the
label is named `scope/name`, no other label with the same `scope/`
prefix can be set on an issue.
The scope is determined by the last occurence of `/`, so for example
`scope/alpha/name` and `scope/beta/name` are considered to be in
different scopes and can coexist.
Exclusive scopes are not enforced by any database rules, however they
are enforced when editing labels at the models level, automatically
removing any existing labels in the same scope when either attaching a
new label or replacing all labels.
In menus use a circle instead of checkbox to indicate they function as
radio buttons per scope. Issue filtering by label ensures that only a
single scoped label is selected at a time. Clicking with alt key can be
used to remove a scoped label, both when editing individual issues and
batch editing.
Label rendering refactor for consistency and code simplification:
* Labels now consistently have the same shape, emojis and tooltips
everywhere. This includes the label list and label assignment menus.
* In label list, show description below label same as label menus.
* Don't use exactly black/white text colors to look a bit nicer.
* Simplify text color computation. There is no point computing luminance
in linear color space, as this is a perceptual problem and sRGB is
closer to perceptually linear.
* Increase height of label assignment menus to show more labels. Showing
only 3-4 labels at a time leads to a lot of scrolling.
* Render all labels with a new RenderLabel template helper function.
Label creation and editing in multiline modal menu:
* Change label creation to open a modal menu like label editing.
* Change menu layout to place name, description and colors on separate
lines.
* Don't color cancel button red in label editing modal menu.
* Align text to the left in model menu for better readability and
consistent with settings layout elsewhere.
Custom exclusive scoped label rendering:
* Display scoped label prefix and suffix with slightly darker and
lighter background color respectively, and a slanted edge between them
similar to the `/` symbol.
* In menus exclusive labels are grouped with a divider line.
---------
Co-authored-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
The main purpose of these home pages should be getting an overview of
what's going on or needs attention. Recently updated is a better default
than newest for that purpose, to avoid missing active issues and pulls
that were not created recently.
The default sorting order in repository issues and pulls remains newest.
Repositories in an organization are already sorted by recently updated.
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Allow back-dating user creation via the `adminCreateUser` API operation.
`CreateUserOption` now has an optional field `created_at`, which can
contain a datetime-formatted string. If this field is present, the
user's `created_unix` database field will be updated to its value.
This is important for Blender's migration of users from Phabricator to
Gitea. There are many users, and the creation timestamp of their account
can give us some indication as to how long someone's been part of the
community.
The back-dating is done in a separate query that just updates the user's
`created_unix` field. This was the easiest and cleanest way I could
find, as in the initial `INSERT` query the field always is set to "now".
Fix#22797.
## Reason
If a comment was migrated from other platforms, this comment may have an
original author and its poster is always not the original author. When
the `roleDescriptor` func get the poster's role descriptor for a
comment, it does not check if the comment has an original author. So the
migrated comments' original authors might be marked as incorrect roles.
---------
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
To avoid duplicated load of the same data in an HTTP request, we can set
a context cache to do that. i.e. Some pages may load a user from a
database with the same id in different areas on the same page. But the
code is hidden in two different deep logic. How should we share the
user? As a result of this PR, now if both entry functions accept
`context.Context` as the first parameter and we just need to refactor
`GetUserByID` to reuse the user from the context cache. Then it will not
be loaded twice on an HTTP request.
But of course, sometimes we would like to reload an object from the
database, that's why `RemoveContextData` is also exposed.
The core context cache is here. It defines a new context
```go
type cacheContext struct {
ctx context.Context
data map[any]map[any]any
lock sync.RWMutex
}
var cacheContextKey = struct{}{}
func WithCacheContext(ctx context.Context) context.Context {
return context.WithValue(ctx, cacheContextKey, &cacheContext{
ctx: ctx,
data: make(map[any]map[any]any),
})
}
```
Then you can use the below 4 methods to read/write/del the data within
the same context.
```go
func GetContextData(ctx context.Context, tp, key any) any
func SetContextData(ctx context.Context, tp, key, value any)
func RemoveContextData(ctx context.Context, tp, key any)
func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error)
```
Then let's take a look at how `system.GetString` implement it.
```go
func GetSetting(ctx context.Context, key string) (string, error) {
return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) {
return cache.GetString(genSettingCacheKey(key), func() (string, error) {
res, err := GetSettingNoCache(ctx, key)
if err != nil {
return "", err
}
return res.SettingValue, nil
})
})
}
```
First, it will check if context data include the setting object with the
key. If not, it will query from the global cache which may be memory or
a Redis cache. If not, it will get the object from the database. In the
end, if the object gets from the global cache or database, it will be
set into the context cache.
An object stored in the context cache will only be destroyed after the
context disappeared.
These functions don't examine contents, just filenames, so they don't
fit in well in a markup module.
This was originally part of
https://github.com/go-gitea/gitea/pull/22177.
Signed-off-by: Nick Guenther <nick.guenther@polymtl.ca>
As discussed in #22847 the helpers in helpers.less need to have a
separate prefix as they are causing conflicts with fomantic styles
This will allow us to have the `.gt-hidden { display:none !important; }`
style that is needed to for the reverted PR.
Of note in doing this I have noticed that there was already a conflict
with at least one chroma style which this PR now avoids.
I've also added in the `gt-hidden` style that matches the tailwind one
and switched the code that needed it to use that.
Signed-off-by: Andrew Thornton <art27@cantab.net>
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Add setting to allow edits by maintainers by default, to avoid having to
often ask contributors to enable this.
This also reorganizes the pull request settings UI to improve clarity.
It was unclear which checkbox options were there to control available
merge styles and which merge styles they correspond to.
Now there is a "Merge Styles" label followed by the merge style options
with the same name as in other menus. The remaining checkboxes were
moved to the bottom, ordered rougly by typical order of operations.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This code was copy-pasted at some point. Revisit it to reunify it.
~~Doing that then encouraged simplifying the types of a couple of
related functions.~~
~~As a follow-up, move two helper functions, `isReadmeFile()` and
`isReadmeFileExtension()`, intimately tied to `findReadmeFile()`, in as
package-private.~~
Signed-off-by: Nick Guenther <nick.guenther@polymtl.ca>
Original Issue: https://github.com/go-gitea/gitea/issues/22102
This addition would be a big benefit for design and art teams using the
issue tracking.
The preview will be the latest "image type" attachments on an issue-
simple, and allows for automatic updates of the cover image as issue
progress is made!
This would make Gitea competitive with Trello... wouldn't it be amazing
to say goodbye to Atlassian products? Ha.
First image is the most recent, the SQL will fetch up to 5 latest images
(URL string).
All images supported by browsers plus upcoming formats: *.avif *.bmp
*.gif *.jpg *.jpeg *.jxl *.png *.svg *.webp
The CSS will try to center-align images until it cannot, then it will
left align with overflow hidden. Single images get to be slightly
larger!
Tested so far on: Chrome, Firefox, Android Chrome, Android Firefox.
Current revision with light and dark themes:
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
In Go code, HTMLURL should be only used for external systems, like
API/webhook/mail/notification, etc.
If a URL is used by `Redirect` or rendered in a template, it should be a
relative URL (aka `Link()` in Gitea)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Fixes#19555
Test-Instructions:
https://github.com/go-gitea/gitea/pull/21441#issuecomment-1419438000
This PR implements the mapping of user groups provided by OIDC providers
to orgs teams in Gitea. The main part is a refactoring of the existing
LDAP code to make it usable from different providers.
Refactorings:
- Moved the router auth code from module to service because of import
cycles
- Changed some model methods to take a `Context` parameter
- Moved the mapping code from LDAP to a common location
I've tested it with Keycloak but other providers should work too. The
JSON mapping format is the same as for LDAP.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
partially fix#19345
This PR add some `Link` methods for different objects. The `Link`
methods are not different from `HTMLURL`, they are lack of the absolute
URL. And most of UI `HTMLURL` have been replaced to `Link` so that users
can visit them from a different domain or IP.
This PR also introduces a new javascript configuration
`window.config.reqAppUrl` which is different from `appUrl` which is
still an absolute url but the domain has been replaced to the current
requested domain.
During the refactoring of the git module, I found there were some
strange operations. This PR tries to fix 2 of them
1. The empty argument `--` in repo_attribute.go, which was introduced by
#16773. It seems unnecessary because nothing else would be added later.
2. The complex git service logic in repo/http.go.
* Before: the `hasAccess` only allow `service == "upload-pack" ||
service == "receive-pack"`
* After: unrelated code is removed. No need to call ToTrustedCmdArgs
anymore.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR fixes two problems. One is when filter repository issues, only
repository level projects are listed. Another is if you list open
issues, only open projects will be displayed in filter options and if
you list closed issues, only closed projects will be displayed in filter
options.
In this PR, both repository level and org/user level projects will be
displayed in filter, and both open and closed projects will be listed as
filter items.
---------
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
Every user can already disable the filter manually, so the explicit
setting is absolutely useless and only complicates the logic.
Previously, there was also unexpected behavior when multiple query
parameters were present.
---------
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR follows #21535 (and replace #22592)
## Review without space diff
https://github.com/go-gitea/gitea/pull/22678/files?diff=split&w=1
## Purpose of this PR
1. Make git module command completely safe (risky user inputs won't be
passed as argument option anymore)
2. Avoid low-level mistakes like
https://github.com/go-gitea/gitea/pull/22098#discussion_r1045234918
3. Remove deprecated and dirty `CmdArgCheck` function, hide the `CmdArg`
type
4. Simplify code when using git command
## The main idea of this PR
* Move the `git.CmdArg` to the `internal` package, then no other package
except `git` could use it. Then developers could never do
`AddArguments(git.CmdArg(userInput))` any more.
* Introduce `git.ToTrustedCmdArgs`, it's for user-provided and already
trusted arguments. It's only used in a few cases, for example: use git
arguments from config file, help unit test with some arguments.
* Introduce `AddOptionValues` and `AddOptionFormat`, they make code more
clear and simple:
* Before: `AddArguments("-m").AddDynamicArguments(message)`
* After: `AddOptionValues("-m", message)`
* -
* Before: `AddArguments(git.CmdArg(fmt.Sprintf("--author='%s <%s>'",
sig.Name, sig.Email)))`
* After: `AddOptionFormat("--author='%s <%s>'", sig.Name, sig.Email)`
## FAQ
### Why these changes were not done in #21535 ?
#21535 is mainly a search&replace, it did its best to not change too
much logic.
Making the framework better needs a lot of changes, so this separate PR
is needed as the second step.
### The naming of `AddOptionXxx`
According to git's manual, the `--xxx` part is called `option`.
### How can it guarantee that `internal.CmdArg` won't be not misused?
Go's specification guarantees that. Trying to access other package's
internal package causes compilation error.
And, `golangci-lint` also denies the git/internal package. Only the
`git/command.go` can use it carefully.
### There is still a `ToTrustedCmdArgs`, will it still allow developers
to make mistakes and pass untrusted arguments?
Generally speaking, no. Because when using `ToTrustedCmdArgs`, the code
will be very complex (see the changes for examples). Then developers and
reviewers can know that something might be unreasonable.
### Why there was a `CmdArgCheck` and why it's removed?
At the moment of #21535, to reduce unnecessary changes, `CmdArgCheck`
was introduced as a hacky patch. Now, almost all code could be written
as `cmd := NewCommand(); cmd.AddXxx(...)`, then there is no need for
`CmdArgCheck` anymore.
### Why many codes for `signArg == ""` is deleted?
Because in the old code, `signArg` could never be empty string, it's
either `-S[key-id]` or `--no-gpg-sign`. So the `signArg == ""` is just
dead code.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Our trace logging is far from perfect and is difficult to follow.
This PR:
* Add trace logging for process manager add and remove.
* Fixes an errant read file for git refs in getMergeCommit
* Brings in the pullrequest `String` and `ColorFormat` methods
introduced in #22568
* Adds a lot more logging in to testPR etc.
Ref #22578
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
The error reported when a user passes a private ssh key as their ssh
public key is not very nice.
This PR improves this slightly.
Ref #22693
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
Fixes#22183
Replaces #22187
This PR adds secrets for users. I refactored the files for organizations
and repos to use the same logic and templates. I splitted the secrets
from deploy keys again and reverted the fix from #22187.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
When using a markdown template the commit body is prepended to text in
the description. This adds the same functionality for yaml templates,
adding the commit message in a field with id "body".
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
And also the other way around, it would show an non-working URL in the
message when pull requests are disabled on the base repository but
enabled on the fork.
Currently only a single project like milestone, not multiple like
labels.
Implements #14298
Code by @brechtvl
---------
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
On activating local accounts, the error message didn't differentiate
between using a wrong or expired token, or a wrong password. The result
could already be obtained from the behaviour (different screens were
presented), but the error message was misleading and lead to confusion
for new users on Codeberg with Forgejo.
Now, entering a wrong password for a valid token prints a different
error message.
The problem was introduced in 0f14f69e60.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
The `commit_id` property name is the same as equivalent functionality in
GitHub. If the action was not caused by a commit, an empty string is
used.
This can for example be used to automatically add a Resolved label to an
issue fixed by a commit, or clear it when the issue is reopened.
The regular login flow can use a `redirect_to` cookie to ensure the user
ends their authentication flow on the same page as where they started
it.
This commit adds the same functionality to the OAuth login URLs, so that
you can use URLs like these to directly use a specific OAuth provider:
`/user/oauth2/{provider}?redirect_to={post-login path}`
Only the `auth.SignInOAuth()` function needed a change for this, as the
rest of the login flow is aware of this cookie and uses it properly
already.
Change the mailer interface to prevent leaking of possible hidden email
addresses when sending to multiple recipients.
Co-authored-by: Gusted <williamzijl7@hotmail.com>
This PR adds the support for scopes of access tokens, mimicking the
design of GitHub OAuth scopes.
The changes of the core logic are in `models/auth` that `AccessToken`
struct will have a `Scope` field. The normalized (no duplication of
scope), comma-separated scope string will be stored in `access_token`
table in the database.
In `services/auth`, the scope will be stored in context, which will be
used by `reqToken` middleware in API calls. Only OAuth2 tokens will have
granular token scopes, while others like BasicAuth will default to scope
`all`.
A large amount of work happens in `routers/api/v1/api.go` and the
corresponding `tests/integration` tests, that is adding necessary scopes
to each of the API calls as they fit.
- [x] Add `Scope` field to `AccessToken`
- [x] Add access control to all API endpoints
- [x] Update frontend & backend for when creating tokens
- [x] Add a database migration for `scope` column (enable 'all' access
to past tokens)
I'm aiming to complete it before Gitea 1.19 release.
Fixes#4300
As suggest by Go developers, use `filepath.WalkDir` instead of
`filepath.Walk` because [*Walk is less efficient than WalkDir,
introduced in Go 1.16, which avoids calling `os.Lstat` on every file or
directory visited](https://pkg.go.dev/path/filepath#Walk).
This proposition address that, in a similar way as
https://github.com/go-gitea/gitea/pull/22392 did.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR introduce glob match for protected branch name. The separator is
`/` and you can use `*` matching non-separator chars and use `**` across
separator.
It also supports input an exist or non-exist branch name as matching
condition and branch name condition has high priority than glob rule.
Should fix#2529 and #15705
screenshots
<img width="1160" alt="image"
src="https://user-images.githubusercontent.com/81045/205651179-ebb5492a-4ade-4bb4-a13c-965e8c927063.png">
Co-authored-by: zeripath <art27@cantab.net>
The API endpoints for "git" can panic if they are called on an empty
repo. We can simply allow empty repos for these endpoints without worry
as they should just work.
Fix#22452
Signed-off-by: Andrew Thornton <art27@cantab.net>
closes#13585fixes#9067fixes#2386
ref #6226
ref #6219fixes#745
This PR adds support to process incoming emails to perform actions.
Currently I added handling of replies and unsubscribing from
issues/pulls. In contrast to #13585 the IMAP IDLE command is used
instead of polling which results (in my opinion 😉) in cleaner code.
Procedure:
- When sending an issue/pull reply email, a token is generated which is
present in the Reply-To and References header.
- IMAP IDLE waits until a new email arrives
- The token tells which action should be performed
A possible signature and/or reply gets stripped from the content.
I added a new service to the drone pipeline to test the receiving of
incoming mails. If we keep this in, we may test our outgoing emails too
in future.
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
- Move the file `compare.go` and `slice.go` to `slice.go`.
- Fix `ExistsInSlice`, it's buggy
- It uses `sort.Search`, so it assumes that the input slice is sorted.
- It passes `func(i int) bool { return slice[i] == target })` to
`sort.Search`, that's incorrect, check the doc of `sort.Search`.
- Conbine `IsInt64InSlice(int64, []int64)` and `ExistsInSlice(string,
[]string)` to `SliceContains[T]([]T, T)`.
- Conbine `IsSliceInt64Eq([]int64, []int64)` and `IsEqualSlice([]string,
[]string)` to `SliceSortedEqual[T]([]T, T)`.
- Add `SliceEqual[T]([]T, T)` as a distinction from
`SliceSortedEqual[T]([]T, T)`.
- Redesign `RemoveIDFromList([]int64, int64) ([]int64, bool)` to
`SliceRemoveAll[T]([]T, T) []T`.
- Add `SliceContainsFunc[T]([]T, func(T) bool)` and
`SliceRemoveAllFunc[T]([]T, func(T) bool)` for general use.
- Add comments to explain why not `golang.org/x/exp/slices`.
- Add unit tests.
After #22362, we can feel free to use transactions without
`db.DefaultContext`.
And there are still lots of models using `db.DefaultContext`, I think we
should refactor them carefully and one by one.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
`code.title` isn't an existing translation.
`explore.code` is the translation used for the tab, which I think
matches closely enough for this instead of a brand new translation.
Open to feedback on whether a new translation would be preferred
instead.
Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Previously, there was an `import services/webhooks` inside
`modules/notification/webhook`.
This import was removed (after fighting against many import cycles).
Additionally, `modules/notification/webhook` was moved to
`modules/webhook`,
and a few structs/constants were extracted from `models/webhooks` to
`modules/webhook`.
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Push mirrors `sync_on_commit` option was added to the web interface in
v1.18.0. However, it's not added to the API. This PR updates the API
endpoint.
Fixes#22267
Also, I think this should be backported to 1.18
This PR changed the Auth interface signature from
`Verify(http *http.Request, w http.ResponseWriter, store DataStore, sess
SessionStore) *user_model.User`
to
`Verify(http *http.Request, w http.ResponseWriter, store DataStore, sess
SessionStore) (*user_model.User, error)`.
There is a new return argument `error` which means the verification
condition matched but verify process failed, we should stop the auth
process.
Before this PR, when return a `nil` user, we don't know the reason why
it returned `nil`. If the match condition is not satisfied or it
verified failure? For these two different results, we should have
different handler. If the match condition is not satisfied, we should
try next auth method and if there is no more auth method, it's an
anonymous user. If the condition matched but verify failed, the auth
process should be stop and return immediately.
This will fix#20563
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Jason Song <i@wolfogre.com>
If user has reached the maximum limit of repositories:
- Before
- disallow create
- allow fork without limit
- This patch:
- disallow create
- disallow fork
- Add option `ALLOW_FORK_WITHOUT_MAXIMUM_LIMIT` (Default **true**) :
enable this allow user fork repositories without maximum number limit
fixed https://github.com/go-gitea/gitea/issues/21847
Signed-off-by: Xinyu Zhou <i@sourcehut.net>
Since we changed the /api/v1/ routes to disallow session authentication
we also removed their reliance on CSRF. However, we left the
ReverseProxy authentication here - but this means that POSTs to the API
are no longer protected by CSRF.
Now, ReverseProxy authentication is a kind of session authentication,
and is therefore inconsistent with the removal of session from the API.
This PR proposes that we simply remove the ReverseProxy authentication
from the API and therefore users of the API must explicitly use tokens
or basic authentication.
Replace #22077Close#22221Close#22077
Signed-off-by: Andrew Thornton <art27@cantab.net>
Fixes#19091
Add Feed for Releases and Tags, can be accessed through
`reponame/releases.rss`, `reponame/releases.atom`, `reponame/tags.rss`,
and `reponame/tags.atom`
Signed-off-by: Reo <reo_999@proton.me>
Fixes#22178
After this change upload versions with different semver metadata are
treated as the same version and trigger a duplicated version error.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
As recognised in #21841 the rendering of plain text files is somewhat
incorrect when there are ambiguous characters as the html code is double
escaped. In fact there are several more problems here.
We have a residual isRenderedHTML which is actually simply escaping the
file - not rendering it. This is badly named and gives the wrong
impression.
There is also unusual behaviour whether the file is called a Readme or
not and there is no way to get to the source code if the file is called
README.
In reality what should happen is different depending on whether the file
is being rendered a README at the bottom of the directory view or not.
1. If it is rendered as a README on a directory - it should simply be
escaped and rendered as `<pre>` text.
2. If it is rendered as a file then it should be rendered as source
code.
This PR therefore does:
1. Rename IsRenderedHTML to IsPlainText
2. Readme files rendered at the bottom of the directory are rendered
without line numbers
3. Otherwise plain text files are rendered as source code.
Replace #21841
Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
depends on #22094
Fixes https://codeberg.org/forgejo/forgejo/issues/77
The old logic did not consider `is_internal`.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Close#14601Fix#3690
Revive of #14601.
Updated to current code, cleanup and added more read/write checks.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andre Bruch <ab@andrebruch.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Norwin <git@nroo.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Gitea will migrate the database model version automatically, but it
should be able to be disabled and keep Gitea shutdown if the version is
not matched.
`hex.EncodeToString` has better performance than `fmt.Sprintf("%x",
[]byte)`, we should use it as much as possible.
I'm not an extreme fan of performance, so I think there are some
exceptions:
- `fmt.Sprintf("%x", func(...)[N]byte())`
- We can't slice the function return value directly, and it's not worth
adding lines.
```diff
func A()[20]byte { ... }
- a := fmt.Sprintf("%x", A())
- a := hex.EncodeToString(A()[:]) // invalid
+ tmp := A()
+ a := hex.EncodeToString(tmp[:])
```
- `fmt.Sprintf("%X", []byte)`
- `strings.ToUpper(hex.EncodeToString(bytes))` has even worse
performance.
Change all license headers to comply with REUSE specification.
Fix#16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
If a deleted-branch has already been restored, a request to restore it
again will cause a NPE. This PR adds detection for this case, but also
disables buttons when they're clicked in order to help prevent
accidental repeat requests.
Fix#21930
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR addresses #19586
I added a mutex to the upload version creation which will prevent the
push errors when two requests try to create these database entries. I'm
not sure if this should be the final solution for this problem.
I added a workaround to allow a reupload of missing blobs. Normally a
reupload is skipped because the database knows the blob is already
present. The workaround checks if the blob exists on the file system.
This should not be needed anymore with the above fix so I marked this
code to be removed with Gitea v1.20.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Enable this to require captcha validation for user login. You also must
enable `ENABLE_CAPTCHA`.
Summary:
- Consolidate CAPTCHA template
- add CAPTCHA handle and context
- add `REQUIRE_CAPTCHA_FOR_LOGIN` config and docs
- Consolidate CAPTCHA set-up and verification code
Partially resolved#6049
Signed-off-by: Xinyu Zhou <i@sourcehut.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Fix#20456
At some point during the 1.17 cycle abbreviated refishs to issue
branches started breaking. This is likely due serious inconsistencies in
our management of refs throughout Gitea - which is a bug needing to be
addressed in a different PR. (Likely more than one)
We should try to use non-abbreviated `fullref`s as much as possible.
That is where a user has inputted a abbreviated `refish` we should add
`refs/heads/` if it is `branch` etc. I know people keep writing and
merging PRs that remove prefixes from stored content but it is just
wrong and it keeps causing problems like this. We should only remove the
prefix at the time of
presentation as the prefix is the only way of knowing umambiguously and
permanently if the `ref` is referring to a `branch`, `tag` or `commit` /
`SHA`. We need to make it so that every ref has the appropriate prefix,
and probably also need to come up with some definitely unambiguous way
of storing `SHA`s if they're used in a `ref` or `refish` field. We must
not store a potentially
ambiguous `refish` as a `ref`. (Especially when referring a `tag` -
there is no reason why users cannot create a `branch` with the same
short name as a `tag` and vice versa and any attempt to prevent this
will fail. You can even create a `branch` and a
`tag` that matches the `SHA` pattern.)
To that end in order to fix this bug, when parsing issue templates check
the provided `Ref` (here a `refish` because almost all users do not know
or understand the subtly), if it does not start with `refs/` add the
`BranchPrefix` to it. This allows people to make their templates refer
to a `tag` but not to a `SHA` directly. (I don't think that is
particularly unreasonable but if people disagree I can make the `refish`
be checked to see if it matches the `SHA` pattern.)
Next we need to handle the issue links that are already written. The
links here are created with `git.RefURL`
Here we see there is a bug introduced in #17551 whereby the provided
`ref` argument can be double-escaped so we remove the incorrect external
escape. (The escape added in #17551 is in the right place -
unfortunately I missed that the calling function was doing the wrong
thing.)
Then within `RefURL()` we check if an unprefixed `ref` (therefore
potentially a `refish`) matches the `SHA` pattern before assuming that
is actually a `commit` - otherwise is assumed to be a `branch`. This
will handle most of the problem cases excepting the very unusual cases
where someone has deliberately written a `branch` to look like a `SHA1`.
But please if something is called a `ref` or interpreted as a `ref` make
it a full-ref before storing or using it. By all means if something is a
`branch` assume the prefix is removed but always add it back in if you
are using it as a `ref`. Stop storing abbreviated `branch` names and
`tag` names - which are `refish` as a `ref`. It will keep on causing
problems like this.
Fix#20456
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This patch provide a mechanism to disable RSS/Atom feed.
Signed-off-by: Xinyu Zhou <i@sourcehut.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
The doctor check `storages` currently only checks the attachment
storage. This PR adds some basic garbage collection functionality for
the other types of storage.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Fix#19513
This PR introduce a new db method `InTransaction(context.Context)`,
and also builtin check on `db.TxContext` and `db.WithTx`.
There is also a new method `db.AutoTx` has been introduced but could be used by other PRs.
`WithTx` will always open a new transaction, if a transaction exist in context, return an error.
`AutoTx` will try to open a new transaction if no transaction exist in context.
That means it will always enter a transaction if there is no error.
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
In #21637 it was mentioned that the purpose of the API routes for the
packages is unclear. This PR adds some documentation.
Fix#21637
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Fix#20921.
The `ctx.Repo.GitRepo` has been used in deleting issues when the issue
is a PR.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
This PR enhances the CORS middleware usage by allowing for the headers
to be configured in `app.ini`.
Fixes#21746
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
The purpose of #18982 is to improve the SMTP mailer, but there were some
unrelated changes made to the SMTP auth in
d60c438694
This PR reverts these unrelated changes, fix#21744
Related #20471
This PR adds global quota limits for the package registry. Settings for
individual users/orgs can be added in a seperate PR using the settings
table.
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This addresses #21707 and adds a second package test case for a
non-semver compatible version (this might be overkill though since you
could also edit the old package version to have an epoch in front and
see the error, this just seemed more flexible for the future).
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Fix https://github.com/go-gitea/gitea/issues/21666
Caused by https://github.com/go-gitea/gitea/pull/19663
Before: when install, the INTERNAL_TOKEN was always generated and saved.
But the internal token may be already there by INTERNAL_TOKEN_URI
After: INTERNAL_TOKEN_URI file must be non-empty. When install, skip
internal token generation if the token exists.
_This is a different approach to #20267, I took the liberty of adapting
some parts, see below_
## Context
In some cases, a weebhook endpoint requires some kind of authentication.
The usual way is by sending a static `Authorization` header, with a
given token. For instance:
- Matrix expects a `Bearer <token>` (already implemented, by storing the
header cleartext in the metadata - which is buggy on retry #19872)
- TeamCity #18667
- Gitea instances #20267
- SourceHut https://man.sr.ht/graphql.md#authentication-strategies (this
is my actual personal need :)
## Proposed solution
Add a dedicated encrypt column to the webhook table (instead of storing
it as meta as proposed in #20267), so that it gets available for all
present and future hook types (especially the custom ones #19307).
This would also solve the buggy matrix retry #19872.
As a first step, I would recommend focusing on the backend logic and
improve the frontend at a later stage. For now the UI is a simple
`Authorization` field (which could be later customized with `Bearer` and
`Basic` switches):
The header name is hard-coded, since I couldn't fine any usecase
justifying otherwise.
## Questions
- What do you think of this approach? @justusbunsi @Gusted @silverwind
- ~~How are the migrations generated? Do I have to manually create a new
file, or is there a command for that?~~
- ~~I started adding it to the API: should I complete it or should I
drop it? (I don't know how much the API is actually used)~~
## Done as well:
- add a migration for the existing matrix webhooks and remove the
`Authorization` logic there
_Closes #19872_
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: delvh <dev.lh@web.de>
I found myself wondering whether a PR I scheduled for automerge was
actually merged. It was, but I didn't receive a mail notification for it
- that makes sense considering I am the doer and usually don't want to
receive such notifications. But ideally I want to receive a notification
when a PR was merged because I scheduled it for automerge.
This PR implements exactly that.
The implementation works, but I wonder if there's a way to avoid passing
the "This PR was automerged" state down so much. I tried solving this
via the database (checking if there's an automerge scheduled for this PR
when sending the notification) but that did not work reliably, probably
because sending the notification happens async and the entry might have
already been deleted. My implementation might be the most
straightforward but maybe not the most elegant.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
A patch to #17335.
Just like AppPath, Gitea writes its own CustomConf into git hook scripts
too. If Gitea's CustomConf changes, then the git push may fail.
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This patch provide a mechanism to disable
[sitemap](https://github.com/go-gitea/gitea/pull/18407).
Signed-off-by: Xinyu Zhou <i@sourcehut.net>
Co-authored-by: 6543 <6543@obermui.de>
Fixes an 500 error/panic if using the changed PR files API with pages
that should return empty lists because there are no items anymore.
`start-end` is then < 0 which ends in panic.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
I noticed an admin is not allowed to upload packages for other users
because `ctx.IsSigned` was not set.
I added a check for `user.IsActive` and `user.ProhibitLogin` too because
both was not checked. Tests enforce this now.
Co-authored-by: Lauris BH <lauris@nix.lv>
The OAuth spec [defines two types of
client](https://datatracker.ietf.org/doc/html/rfc6749#section-2.1),
confidential and public. Previously Gitea assumed all clients to be
confidential.
> OAuth defines two client types, based on their ability to authenticate
securely with the authorization server (i.e., ability to
> maintain the confidentiality of their client credentials):
>
> confidential
> Clients capable of maintaining the confidentiality of their
credentials (e.g., client implemented on a secure server with
> restricted access to the client credentials), or capable of secure
client authentication using other means.
>
> **public
> Clients incapable of maintaining the confidentiality of their
credentials (e.g., clients executing on the device used by the resource
owner, such as an installed native application or a web browser-based
application), and incapable of secure client authentication via any
other means.**
>
> The client type designation is based on the authorization server's
definition of secure authentication and its acceptable exposure levels
of client credentials. The authorization server SHOULD NOT make
assumptions about the client type.
https://datatracker.ietf.org/doc/html/rfc8252#section-8.4
> Authorization servers MUST record the client type in the client
registration details in order to identify and process requests
accordingly.
Require PKCE for public clients:
https://datatracker.ietf.org/doc/html/rfc8252#section-8.1
> Authorization servers SHOULD reject authorization requests from native
apps that don't use PKCE by returning an error message
Fixes#21299
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This stype name is also used in many repos, example:
[``README_ZH.md``](https://github.com/go-gitea/gitea/blob/main/README_ZH.md)
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
According to the OAuth spec
https://datatracker.ietf.org/doc/html/rfc6749#section-6 when "Refreshing
an Access Token"
> The authorization server MUST ... require client authentication for
confidential clients
Fixes#21418
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Previously mentioning a user would link to its profile, regardless of
whether the user existed. This change tests if the user exists and only
if it does - a link to its profile is added.
* Fixes#3444
Signed-off-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This prevents a 500 response, because null pointer exceptions in
rendering the template.
This happends bc the repoId is not in the repoMap because it is delete
fix#19076
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
At the moment, If admin disable Packages, still show the Packages on the
admin dashboard
This patch added a check to hide the Packages entry
Signed-off-by: baronbunny <its@baronbunny.cn>
Signed-off-by: baronbunny <its@baronbunny.cn>
At the moment a repository reference is needed for webhooks. With the
upcoming package PR we need to send webhooks without a repository
reference. For example a package is uploaded to an organization. In
theory this enables the usage of webhooks for future user actions.
This PR removes the repository id from `HookTask` and changes how the
hooks are processed (see `services/webhook/deliver.go`). In a follow up
PR I want to remove the usage of the `UniqueQueue´ and replace it with a
normal queue because there is no reason to be unique.
Co-authored-by: 6543 <6543@obermui.de>
depends on #18871
Added some api integration tests to help testing of #18798.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Fixes#21379
The commits are capped by `setting.UI.FeedMaxCommitNum` so
`len(commits)` is not the correct number. So this PR adds a new
`TotalCommits` field.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Support OAuth2 applications created by admins on the admin panel, they
aren't owned by anybody.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
NuGet symbol file lookup returned 404 on Visual Studio 2019 due to
case-sensitive api router. The api router should accept case-insensitive GUID.
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Partially fix#21213.
This PR will get client IP address from SSH_CONNECTION env which should
be the first field of that. And deliver it to the internal API so Gitea
routers could record the real IP from SSH requests.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
Adds the settings pages to create OAuth2 apps also to the org settings
and allows to create apps for orgs.
Refactoring: the oauth2 related templates are shared for
instance-wide/org/user, and the backend code uses `OAuth2CommonHandlers`
to share code for instance-wide/org/user.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Close#20315 (fix the panic when parsing invalid input), Speed up #20231 (use ls-tree without size field)
Introduce ListEntriesRecursiveFast (ls-tree without size) and ListEntriesRecursiveWithSize (ls-tree with size)
Fixes#21250
Related #20414
Conan packages don't have to follow SemVer.
The migration fixes the setting for all existing Conan and Generic
(#20414) packages.
This fixes error "unauthorized_client: invalid client secret" when
client includes secret in Authorization header rather than request body.
OAuth spec permits both.
Sanity validation that client id and client secret in request are
consistent with Authorization header.
Improve error descriptions. Error codes remain the same.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Calls to ToCommit are very slow due to fetching diffs, analyzing files.
This patch lets us supply `stat` as false to speed fetching a commit
when we don't need the diff.
/v1/repo/commits has a default `stat` set as true now. Set to false to
experience fetching thousands of commits per second instead of 2-5 per
second.
Only load SECRET_KEY and INTERNAL_TOKEN if they exist.
Never write the config file if the keys do not exist, which was only a fallback for Gitea upgraded from < 1.5
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Adds GitHub-like pages to view watched repos and subscribed issues/PRs
This is my second try to fix this, but it is better than the first since
it doesn't uses a filter option which could be slow when accessing
`/issues` or `/pulls` and it shows both pulls and issues (the first try
is #17053).
Closes#16111
Replaces and closes#17053
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This adds an api endpoint `/files` to PRs that allows to get a list of changed files.
built upon #18228, reviews there are included
closes https://github.com/go-gitea/gitea/issues/654
Co-authored-by: Anton Bracke <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fixes#21282
As suggested by the [OAuth RFC](https://www.rfc-editor.org/rfc/rfc6749)
(quoted below), it's helpful to give more detail in the description
> error_description
OPTIONAL. Human-readable ASCII
[[USASCII](https://www.rfc-editor.org/rfc/rfc6749#ref-USASCII)] text
providing **additional information, used to assist the client developer
in understanding the error that occurred.**
Values for the "error_description" parameter MUST NOT include characters
outside the set %x20-21 / %x23-5B / %x5D-7E.
This PR would presumably
Fix#20522Fix#18773Fix#19069Fix#21077Fix#13622
-----
1. Check whether unit type is currently enabled
2. Check if it _will_ be enabled via opt
3. Allow modification as necessary
Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Addition to #20734, Fixes#20717
The `/index.json` endpoint needs to be accessible even if the registry
is private. The NuGet client uses this endpoint without
authentification.
The old fix only works if the NuGet cli is used with `--source <name>`
but not with `--source <url>/index.json`.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Close#20098, in the NPM registry API, implemented to match what's described by https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md#get-v1search
Currently have only implemented the bare minimum to work with the [Unity Package Manager](https://docs.unity3d.com/Manual/upm-ui.html).
Co-authored-by: Jack Vine <jackv@jack-lemur-suse.cat-prometheus.ts.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fixes#21184
Regression of #19552
Instead of using `GetBlobByPath` I use the already existing instances.
We need more information from #19530 if that error is still present.
If you are create a new new branch while viewing file or directory, you
get redirected to the root of the repo. With this PR, you keep your
current path instead of getting redirected to the repo root.
In #21088 I accidentally forgot to support multiple branches. It always
checks the default branch, no matter on which branch you are working on.
With this fix, it always shows the error from the current branch. Sorry
for that.
The PyPI name regexp is too restrictive and only permits lowercase characters. This PR adjusts the regexp to add in support for uppercase characters.
Fix#21014
Add support for triggering webhook notifications on wiki changes.
This PR contains frontend and backend for webhook notifications on wiki actions (create a new page, rename a page, edit a page and delete a page). The frontend got a new checkbox under the Custom Event -> Repository Events section. There is only one checkbox for create/edit/rename/delete actions, because it makes no sense to separate it and others like releases or packages follow the same schema.
The actions itself are separated, so that different notifications will be executed (with the "action" field). All the webhook receivers implement the new interface method (Wiki) and the corresponding tests.
When implementing this, I encounter a little bug on editing a wiki page. Creating and editing a wiki page is technically the same action and will be handled by the ```updateWikiPage``` function. But the function need to know if it is a new wiki page or just a change. This distinction is done by the ```action``` parameter, but this will not be sent by the frontend (on form submit). This PR will fix this by adding the ```action``` parameter with the values ```_new``` or ```_edit```, which will be used by the ```updateWikiPage``` function.
I've done integration tests with matrix and gitea (http).
Fix#16457
Signed-off-by: Aaron Fischer <mail@aaron-fischer.net>
A testing cleanup.
This pull request replaces `os.MkdirTemp` with `t.TempDir`. We can use the `T.TempDir` function from the `testing` package to create temporary directory. The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete.
This saves us at least 2 lines (error check, and cleanup) on every instance, or in some cases adds cleanup that we forgot.
Reference: https://pkg.go.dev/testing#T.TempDir
```go
func TestFoo(t *testing.T) {
// before
tmpDir, err := os.MkdirTemp("", "")
require.NoError(t, err)
defer os.RemoveAll(tmpDir)
// now
tmpDir := t.TempDir()
}
```
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
When migrating add several more important sanity checks:
* SHAs must be SHAs
* Refs must be valid Refs
* URLs must be reasonable
Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <matti@mdranta.net>
The webhook payload should use the right ref when it‘s specified in the testing request.
The compare URL should not be empty, a URL like `compare/A...A` seems useless in most cases but is helpful when testing.
* feat: extend issue template for yaml
* feat: support yaml template
* feat: render form to markdown
* feat: support yaml template for pr
* chore: rename to Fields
* feat: template unmarshal
* feat: split template
* feat: render to markdown
* feat: use full name as template file name
* chore: remove useless file
* feat: use dropdown of fomantic ui
* feat: update input style
* docs: more comments
* fix: render text without render
* chore: fix lint error
* fix: support use description as about in markdown
* fix: add field class in form
* chore: generate swagger
* feat: validate template
* feat: support is_nummber and regex
* test: fix broken unit tests
* fix: ignore empty body of md template
* fix: make multiple easymde editors work in one page
* feat: better UI
* fix: js error in pr form
* chore: generate swagger
* feat: support regex validation
* chore: generate swagger
* fix: refresh each markdown editor
* chore: give up required validation
* fix: correct issue template candidates
* fix: correct checkboxes style
* chore: ignore .hugo_build.lock in docs
* docs: separate out a new doc for merge templates
* docs: introduce syntax of yaml template
* feat: show a alert for invalid templates
* test: add case for a valid template
* fix: correct attributes of required checkbox
* fix: add class not-under-easymde for dropzone
* fix: use more back-quotes
* chore: remove translation in zh-CN
* fix EasyMDE statusbar margin
* fix: remove repeated blocks
* fix: reuse regex for quotes
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* fix hard-coded timeout and error panic in API archive download endpoint
This commit updates the `GET /api/v1/repos/{owner}/{repo}/archive/{archive}`
endpoint which prior to this PR had a couple of issues.
1. The endpoint had a hard-coded 20s timeout for the archiver to complete after
which a 500 (Internal Server Error) was returned to client. For a scripted
API client there was no clear way of telling that the operation timed out and
that it should retry.
2. Whenever the timeout _did occur_, the code used to panic. This was caused by
the API endpoint "delegating" to the same call path as the web, which uses a
slightly different way of reporting errors (HTML rather than JSON for
example).
More specifically, `api/v1/repo/file.go#GetArchive` just called through to
`web/repo/repo.go#Download`, which expects the `Context` to have a `Render`
field set, but which is `nil` for API calls. Hence, a `nil` pointer error.
The code addresses (1) by dropping the hard-coded timeout. Instead, any
timeout/cancelation on the incoming `Context` is used.
The code addresses (2) by updating the API endpoint to use a separate call path
for the API-triggered archive download. This avoids producing HTML-errors on
errors (it now produces JSON errors).
Signed-off-by: Peter Gardfjäll <peter.gardfjall.work@gmail.com>
The recovery, API, Web and package frameworks all create their own HTML
Renderers. This increases the memory requirements of Gitea
unnecessarily with duplicate templates being kept in memory.
Further the reloading framework in dev mode for these involves locking
and recompiling all of the templates on each load. This will potentially
hide concurrency issues and it is inefficient.
This PR stores the templates renderer in the context and stores this
context in the NormalRoutes, it then creates a fsnotify.Watcher
framework to watch files.
The watching framework is then extended to the mailer templates which
were previously not being reloaded in dev.
Then the locales are simplified to a similar structure.
Fix#20210Fix#20211Fix#20217
Signed-off-by: Andrew Thornton <art27@cantab.net>
Adds a new option to only show relevant repo's on the explore page, for bigger Gitea instances like Codeberg this is a nice option to enable to make the explore page more populated with unique and "high" quality repo's. A note is shown that the results are filtered and have the possibility to see the unfiltered results.
Co-authored-by: vednoc <vednoc@protonmail.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
Add code to test if GetAttachmentByID returns an ErrAttachmentNotExist error
and return NotFound instead of InternalServerError
Fix#20884
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Related to #20650.
This will fix the behavior of the single repo action for pull requests and disables the button for other screens that don't have a single repo action currently.
- Currently the function takes in the `UserID` option, but isn't being
used within the SQL query. This patch fixes that by checking that only
teams are being returned that the user belongs to.
Fix#20829
Co-authored-by: delvh <dev.lh@web.de>
Whilst looking at #20840 I noticed that the Mirrors data doesn't appear
to be being used therefore we can remove this and in fact none of the
related code is used elsewhere so it can also be removed.
Related #20840
Related #20804
Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use the total issue count for UI
This fixes a problem where the "All" line item on the Issues or Pull Requests page was only showing the count of the selected repos instead of the total of all issues/prs in all repos.
The "total number of shown issues" number is now stashed in a different context variable in case it wants to be used by the frontend later. It's currently not being used.
Fixes#20574
* Remove unused context variable
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This adds support for getting the user's full name from the reverse
proxy in addition to username and email.
Tested locally with caddy serving as reverse proxy with Tailscale
authentication.
Signed-off-by: Will Norris <will@tailscale.com>
Signed-off-by: Will Norris <will@tailscale.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
The use of `--follow` makes getting these commits very slow on large repositories
as it results in searching the whole commit tree for a blob.
Now as nice as the results of `--follow` are, I am uncertain whether it is really
of sufficient importance to keep around.
Fix#20764
Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This PR rewrites the invisible unicode detection algorithm to more
closely match that of the Monaco editor on the system. It provides a
technique for detecting ambiguous characters and relaxes the detection
of combining marks.
Control characters are in addition detected as invisible in this
implementation whereas they are not on monaco but this is related to
font issues.
Close#19913
Signed-off-by: Andrew Thornton <art27@cantab.net>
* merge `CheckLFSVersion` into `InitFull` (renamed from `InitWithSyncOnce`)
* remove the `Once` during git init, no data-race now
* for doctor sub-commands, `InitFull` should only be called in initialization stage
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This bug affects tests which are sending emails (#20307). Some tests reinitialise the web routes (like `TestNodeinfo`) which messed up the mail templates. There is no reason why the templates should be loaded in the routes method.
* Added support for Pub packages.
* Update docs/content/doc/packages/overview.en-us.md
Co-authored-by: Gergely Nagy <algernon@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Gergely Nagy <algernon@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
This enables git.Command's Run to optionally use the given context directly so its deadline will be respected. Otherwise, it falls back to the previous behavior of using the supplied timeout or a default timeout value of 360 seconds.
repo's serviceRPC() calls now use the context's deadline (which is unset/unlimited) instead of the default 6-minute timeout. This means that large repo clones will no longer arbitrarily time out on the upload-pack step, and pushes can take longer than 6 minutes on the receive-pack step.
Fixes#20680
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* `PROTOCOL`: can be smtp, smtps, smtp+startls, smtp+unix, sendmail, dummy
* `SMTP_ADDR`: domain for SMTP, or path to unix socket
* `SMTP_PORT`: port for SMTP; defaults to 25 for `smtp`, 465 for `smtps`, and 587 for `smtp+startls`
* `ENABLE_HELO`, `HELO_HOSTNAME`: reverse `DISABLE_HELO` to `ENABLE_HELO`; default to false + system hostname
* `FORCE_TRUST_SERVER_CERT`: replace the unclear `SKIP_VERIFY`
* `CLIENT_CERT_FILE`, `CLIENT_KEY_FILE`, `USE_CLIENT_CERT`: clarify client certificates here
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Support localized README
* Slightly simplify getting the readme file and add some tests. Ensure that i18n also
works for docs/ etc.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update modules/markup/renderer.go
* Update modules/markup/renderer.go
* Update modules/markup/renderer.go
Co-authored-by: Andrew Thornton <art27@cantab.net>
- Add a new push mirror to specific repository
- Sync now ( send all the changes to the configured push mirrors )
- Get list of all push mirrors of a repository
- Get a push mirror by ID
- Delete push mirror by ID
Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
WebAuthn have updated their specification to set the maximum size of the
CredentialID to 1023 bytes. This is somewhat larger than our current
size and therefore we need to migrate.
The PR changes the struct to add CredentialIDBytes and migrates the CredentialID string
to the bytes field before another migration drops the old CredentialID field. Another migration
renames this field back.
Fix#20457
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add latest commit's SHA to content response
- When requesting the contents of a filepath, add the latest commit's
SHA to the requested file.
- Resolves#12840
* Add swagger
* Fix NPE
* Fix tests
* Hook into LastCommitCache
* Move AddLastCommitCache to a common nogogit and gogit file
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent NPE
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
- Always respect the user's configured mime type map
- Allow more types like image/pdf/video/audio to serve with correct content-type
- Shorten cache duration of raw files to 5 minutes, matching GitHub
- Don't set `content-disposition: attachment`, let the browser decide whether it wants to download or display a file directly
- Implement rfc5987 for filenames, remove previous hack. Confirmed it working in Safari.
- Make PDF attachment work in Safari by removing `sandbox` attribute.
This change will make a lot more file types open directly in browser now. Logic should generally be more readable than before with less `if` nesting and such.
Replaces: https://github.com/go-gitea/gitea/pull/20460
Replaces: https://github.com/go-gitea/gitea/pull/20455
Fixes: https://github.com/go-gitea/gitea/issues/20404
There are existing packages out there whose version do not conform to SemVer, yet, one would like to have them available in a generic package repository. To this end, remove the SemVer restriction on package versions when using the Generic package registry, and replace it with a check that simply makes sure the version isn't empty.
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
Sometimes users want to receive email notifications of messages they create or reply to,
Added an option to personal preferences to allow users to choose
Closes#20149
The LastCommitCache code is a little complex and there is unnecessary
duplication between the gogit and nogogit variants.
This PR adds the LastCommitCache as a field to the git.Repository and
pre-creates it in the ReferencesGit helpers etc. There has been some
simplification and unification of the variant code.
Signed-off-by: Andrew Thornton <art27@cantab.net>
- When a repository has a pull request template, it will always override
the current content. With this PR it will prepend content to the
template content when appropriate. This is similar how GitHub(and GitLab
I presume) does it and it saves developers time to not go open their
commit and copy paste their will written commit message.
`no-transform` allegedly disables CloudFlare auto-minify and we did not
set caching headers on html or api requests, which seems good to have
regardless.
Transformation is still allowed for asset requests.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
When you create a new release(e.g. via Tea) and specify a tag that already exists on
the repository, Gitea will instead use the `UpdateRelease`
functionality. However it currently doesn't set the Target field. This
PR fixes that.
Cron will try to run certain things at startup but these depend on multiple things
being set-up. Therefore we should initialize cron last.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Examining Organization membership should not necessarily require sign-in if the organization is public and the members are public. Therefore we should adjust `/org/{org}/members` to not require login.
Fix#7501
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: zeripath <art27@cantab.net>
When viewing a subdirectory and the latest commit to that directory in
the table, the commit status icon incorrectly showed the status of the
HEAD commit instead of the latest for that directory.
[spectral](https://github.com/stoplightio/spectral) lints
openapi/swagger files for mistakes of which it has identified a few and
which I've fixed.
I had to put it into `lint-frontend` because it depends on node_modules
so can not run on Drone during the backend target. I plan to refactor
these targets later to `lint-js` and `lint-go` so that they are
categorized based on the tool dependencies.
Support synchronizing with the push mirrors whenever new commits are pushed or synced from pull mirror.
Related Issues: #18220
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Users who are following or being followed by a user should only be
displayed if the viewing user can see them.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Check if project has the same repository id with issue when assign project to issue
* Check if issue's repository id match project's repository id
* Add more permission checking
* Remove invalid argument
* Fix errors
* Add generic check
* Remove duplicated check
* Return error + add check for new issues
* Apply suggestions from code review
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
* Refactor `i18n` to `locale`
- Currently we're using the `i18n` variable naming for the `locale`
struct. This contains locale's specific information and cannot be used
for general i18n purpose, therefore refactoring it to `locale` makes
more sense.
- Ref: https://github.com/go-gitea/gitea/pull/20096#discussion_r906699200
* Update routers/install/install.go
- Completely remove U2F support from 1.18.0, 1.17.0 will be the last
release that U2F is somewhat supported. Users who used U2F would already
be warned about using U2F for a while now and should hopefully already
be migrated. But starting 1.18 definitely remove it.
* Prototyping
* Start work on creating offsets
* Modify tests
* Start prototyping with actual MPH
* Twiddle around
* Twiddle around comments
* Convert templates
* Fix external languages
* Fix latest translation
* Fix some test
* Tidy up code
* Use simple map
* go mod tidy
* Move back to data structure
- Uses less memory by creating for each language a map.
* Apply suggestions from code review
Co-authored-by: delvh <dev.lh@web.de>
* Add some comments
* Fix tests
* Try to fix tests
* Use en-US as defacto fallback
* Use correct slices
* refactor (#4)
* Remove TryTr, add log for missing translation key
* Refactor i18n
- Separate dev and production locale stores.
- Allow for live-reloading in dev mode.
Co-authored-by: zeripath <art27@cantab.net>
* Fix live-reloading & check for errors
* Make linter happy
* live-reload with periodic check (#5)
* Fix tests
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
This PR adds a new manager command to switch on SQL logging and to turn it off.
```
gitea manager logging log-sql
gitea manager logging log-sql --off
```
Signed-off-by: Andrew Thornton <art27@cantab.net>
The code introduced by #18185 gets the error from response after it was processed by goth.
That is incorrect, as goth (and golang.org/x/oauth) doesn't really care about the error, and it sends a token request with an empty authorization code to the server anyway, which always results in a `oauth2: cannot fetch token: 400 Bad Request` error from goth.
It means that unless the "state" parameter is omitted from the error response (which is required to be present, according to [RFC 6749, Section 4.1.2.1](https://datatracker.ietf.org/doc/html/rfc6749#section-4.1.2.1)) or the page is reloaded (makes the session invalid), a 500 Internal Server Error page will be displayed.
This fixes it by handling the error before the request is passed to goth.
* go.mod: add go-fed/{httpsig,activity/pub,activity/streams} dependency
go get github.com/go-fed/activity/streams@master
go get github.com/go-fed/activity/pub@master
go get github.com/go-fed/httpsig@master
* activitypub: implement /api/v1/activitypub/user/{username} (#14186)
Return informations regarding a Person (as defined in ActivityStreams
https://www.w3.org/TR/activitystreams-vocabulary/#dfn-person).
Refs: https://github.com/go-gitea/gitea/issues/14186
Signed-off-by: Loïc Dachary <loic@dachary.org>
* activitypub: add the public key to Person (#14186)
Refs: https://github.com/go-gitea/gitea/issues/14186
Signed-off-by: Loïc Dachary <loic@dachary.org>
* activitypub: go-fed conformant Clock instance
Signed-off-by: Loïc Dachary <loic@dachary.org>
* activitypub: signing http client
Signed-off-by: Loïc Dachary <loic@dachary.org>
* activitypub: implement the ReqSignature middleware
Signed-off-by: Loïc Dachary <loic@dachary.org>
* activitypub: hack_16834
Signed-off-by: Loïc Dachary <loic@dachary.org>
* Fix CI checks-backend errors with go mod tidy
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Change 2021 to 2022, properly format package imports
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Run make fmt and make generate-swagger
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Use Gitea JSON library, add assert for pkp
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Run make fmt again, fix err var redeclaration
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Remove LogSQL from ActivityPub person test
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Assert if json.Unmarshal succeeds
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Cleanup, handle invalid usernames for ActivityPub person GET request
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Rename hack_16834 to user_settings
Signed-off-by: Anthony Wang <ta180m@pm.me>
* Use the httplib module instead of http for GET requests
* Clean up whitespace with make fmt
* Use time.RFC1123 and make the http.Client proxy-aware
* Check if digest algo is supported in setting module
* Clean up some variable declarations
* Remove unneeded copy
* Use system timezone instead of setting.DefaultUILocation
* Use named constant for httpsigExpirationTime
* Make pubKey IRI #main-key instead of /#main-key
* Move /#main-key to #main-key in tests
* Implemented Webfinger endpoint.
* Add visible check.
* Add user profile as alias.
* Add actor IRI and remote interaction URL to WebFinger response
* fmt
* Fix lint errors
* Use go-ap instead of go-fed
* Run go mod tidy to fix missing modules in go.mod and go.sum
* make fmt
* Convert remaining code to go-ap
* Clean up go.sum
* Fix JSON unmarshall error
* Fix CI errors by adding @context to Person() and making sure types match
* Correctly decode JSON in api_activitypub_person_test.go
* Force CI rerun
* Fix TestActivityPubPersonInbox segfault
* Fix lint error
* Use @mariusor's suggestions for idiomatic go-ap usage
* Correctly add inbox/outbox IRIs to person
* Code cleanup
* Remove another LogSQL from ActivityPub person test
* Move httpsig algos slice to an init() function
* Add actor IRI and remote interaction URL to WebFinger response
* Update TestWebFinger to check for ActivityPub IRI in aliases
* make fmt
* Force CI rerun
* WebFinger: Add CORS header and fix Href -> Template for remote interactions
The CORS header is needed due to https://datatracker.ietf.org/doc/html/rfc7033#section-5 and fixes some Peertube <-> Gitea federation issues
* make lint-backend
* Make sure Person endpoint has Content-Type application/activity+json and includes PreferredUsername, URL, and Icon
Setting the correct Content-Type is essential for federating with Mastodon
* Use UTC instead of GMT
* Rename pkey to pubKey
* Make sure HTTP request Date in GMT
* make fmt
* dont drop err
* Make sure API responses always refer to username in original case
Copied from what I wrote on #19133 discussion: Handling username case is a very tricky issue and I've already encountered a Mastodon <-> Gitea federation bug due to Gitea considering Ta180m and ta180m to be the same user while Mastodon thinks they are two different users. I think the best way forward is for Gitea to only use the original case version of the username for federation so other AP software don't get confused.
* Move httpsig algs constant slice to modules/setting/federation.go
* Add new federation settings to app.example.ini and config-cheat-sheet
* Return if marshalling error
* Make sure Person IRIs are generated correctly
This commit ensures that if the setting.AppURL is something like "http://127.0.0.1:42567" (like in the integration tests), a trailing slash will be added after that URL.
* If httpsig verification fails, fix Host header and try again
This fixes a very rare bug when Gitea and another AP server (confirmed to happen with Mastodon) are running on the same machine, Gitea fails to verify incoming HTTP signatures. This is because the other AP server creates the sig with the public Gitea domain as the Host. However, when Gitea receives the request, the Host header is instead localhost, so the signature verification fails. Manually changing the host header to the correct value and trying the veification again fixes the bug.
* Revert "If httpsig verification fails, fix Host header and try again"
This reverts commit f53e46c721.
The bug was actually caused by nginx messing up the Host header when reverse-proxying since I didn't have the line `proxy_set_header Host $host;` in my nginx config for Gitea.
* Go back to using ap.IRI to generate inbox and outbox IRIs
* use const for key values
* Update routers/web/webfinger.go
* Use ctx.JSON in Person response to make code cleaner
* Revert "Use ctx.JSON in Person response to make code cleaner"
This doesn't work because the ctx.JSON() function already sends the response out and it's too late to edit the headers.
This reverts commit 95aad98897.
* Use activitypub.ActivityStreamsContentType for Person response Content Type
* Limit maximum ActivityPub request and response sizes to a configurable setting
* Move setting key constants to models/user/setting_keys.go
* Fix failing ActivityPubPerson integration test by checking the correct field for username
* Add a warning about changing settings that can break federation
* Add better comments
* Don't multiply Federation.MaxSize by 1<<20 twice
* Add more better comments
* Fix failing ActivityPubMissingPerson test
We now use ctx.ContextUser so the message printed out when a user does not exist is slightly different
* make generate-swagger
For some reason I didn't realize that /templates/swagger/v1_json.tmpl was machine-generated by make generate-swagger... I've been editing it by hand for three months! 🤦
* Move getting the RFC 2616 time to a separate function
* More code cleanup
* Update go-ap to fix empty liked collection and removed unneeded HTTP headers
* go mod tidy
* Add ed25519 to httpsig algorithms
* Use go-ap/jsonld to add @context and marshal JSON
* Change Gitea user agent from the default to Gitea/Version
* Use ctx.ServerError and remove all remote interaction code from webfinger.go
If there are dangling stopwatches with missing issues there will be repeated
logging of Unable to APIFormat stopwatches. These are unhelpful and instead
we should only log if the error is not an issue not exist error.
And we should also prevent an error on missing issue in GetActiveStopwatch too
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Allow render HTML with css/js external links
* Fix bug because of filename escape chars
* Fix lint
* Update docs about new configuration item
* Fix bug of render HTML in sub directory
* Add CSP head for displaying iframe in rendering file
* Fix test
* Apply suggestions from code review
Co-authored-by: delvh <dev.lh@web.de>
* Some improvements
* some improvement
* revert change in SanitizerDisabled of external renderer
* Add sandbox for iframe and support allow-scripts and allow-same-origin
* refactor
* fix
* fix lint
* fine tune
* use single option RENDER_CONTENT_MODE, use sandbox=allow-scripts
* fine tune CSP
* Apply suggestions from code review
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* When non-admin users use code search, get code unit accessible repos in one main query
* Modified some comments to match the changes
* Removed unnecessary check for Access Mode in Collaboration table
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Move access and repo permission to models/perm/access
* fix test
* fix git test
* Move functions sequence
* Some improvements per @KN4CK3R and @delvh
* Move issues related code to models/issues
* Move some issues related sub package
* Merge
* Fix test
* Fix test
* Fix test
* Fix test
* Rename some files
* Move access and repo permission to models/perm/access
* fix test
* Move some git related files into sub package models/git
* Fix build
* fix git test
* move lfs to sub package
* move more git related functions to models/git
* Move functions sequence
* Some improvements per @KN4CK3R and @delvh
Automatically add sidebar in the wiki view containing a TOC for the wiki page.
Make the TOC collapsable
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Ensure minimum mirror interval is reported on settings page
Expecting users to guess the minimum mirror interval appears a little unkind.
In this PR we simply change the locale string to include the minimum interval.
This will of course be affected by our current localization framework but...
we can fix that else where.
This PR also includes some fixes for error handling on the settings page as
previously the mirror block amongst others would simply disappear on error.
Fix#3737
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update options/locale/locale_en-US.ini
Co-authored-by: Gusted <williamzijl7@hotmail.com>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update options/locale/locale_en-US.ini
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Move some repository related code into sub package
* Move more repository functions out of models
* Fix lint
* Some performance optimization for webhooks and others
* some refactors
* Fix lint
* Fix
* Update modules/repository/delete.go
Co-authored-by: delvh <dev.lh@web.de>
* Fix test
* Merge
* Fix test
* Fix test
* Fix test
* Fix test
Co-authored-by: delvh <dev.lh@web.de>
Fixes#12338
This allows use to talk to the API with our ssh certificate (and/or ssh-agent) without needing to fetch an API key or tokens.
It will just automatically work when users have added their ssh principal in gitea.
This needs client code in tea
Update: also support normal pubkeys
ref: https://tools.ietf.org/html/draft-cavage-http-signatures
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Fixes#19791 by adding an check if filtering after any repo; if yes, simply set the total count for the pageing to the sum of the issue count for each selected repo by utilize `issueCountByRepo`.
Fix#19791
- To use the web's API to get information about a issue/pull on a
repository, doesn't require authorization(nor that the repository isn't
archived).
- Regressed by: #19318
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Milestones in archived repos should not be displayed on `/milestones`. Therefore
we should exclude these repositories from milestones page.
Fix#18257
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add LFS API
* Update routers/api/v1/repo/file.go
Co-authored-by: Gusted <williamzijl7@hotmail.com>
* Apply suggestions
* Apply suggestions
* Update routers/api/v1/repo/file.go
Co-authored-by: Gusted <williamzijl7@hotmail.com>
* Report errors
* ADd test
* Use own repo for test
* Use different repo name
* Improve handling
* Slight restructures
1. Avoid reading the blob data multiple times
2. Ensure that caching is only checked when about to serve the blob/lfs
3. Avoid nesting by returning early
4. Make log message a bit more clear
5. Ensure that the dataRc is closed by defer when passed to ServeData
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
If the context is cancelled `.NotificationUnreadCount` in a template can
cause an infinite loop with `ctx.ServerError()` being called, which
creates a template that then calls `.NotificationUnreadCount` calling
`GetNotificationCount()` with the cancelled context resulting in an
error that calls `ctx.ServerError`... and so on...
This PR simply stops calling `ctx.ServerError` in the error handler code
for `.NotificationUnreadCount` as we have already started rendering and
so it is too late to call `ctx.ServerError`. Additionally we skip
logging the error if it's a context cancelled error.
Fix#19793
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
The cache service can be disabled - at which point ctx.Cache will be nil
and the use of it will cause an NPE.
The main part of this PR is that the cache is used for restricting
resending of activation mails and without this we cache we cannot
restrict this. Whilst this code could be re-considered to use the db and
probably should be, I think we can simply disable this code in the case
that the cache is disabled.
There are also several bug fixes in the /nodeinfo API endpoint.
Signed-off-by: Andrew Thornton <art27@cantab.net>
use http.StatusTemporaryRedirect(307) when serve avatar directly
browser caches 301 redirections, pre-signed s3 url would expire at some later point
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
- Don't use hacky solution to limit to the correct RepoID's, instead use
current code to handle these limits. The existing code is more correct
than the hacky solution.
- Resolves#19636
- Add test-case
* Update go tool dependencies
Updated all tool dependencies to latest tags, hoping CI will like it.
* fix new lint errors
* handle more strings.Title cases
* remove lint skip
This adds the [Webfinger](https://webfinger.net/) endpoint for federation.
Supported schemes are `acct` and `mailto`. The profile and avatar url are returned as metadata.
Although the use of LastModified dates for caching of git objects should be
discouraged (as it is not native to git - and there are a LOT of ways this
could be incorrect) - LastModified dates can be a helpful somewhat more human
way of caching for simple cases.
This PR adds this header and handles the If-Modified-Since header to the /raw/
routes.
Fix#18354
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Fix indention
Signed-off-by: kolaente <k@knt.li>
* Add option to merge a pr right now without waiting for the checks to succeed
Signed-off-by: kolaente <k@knt.li>
* Fix lint
Signed-off-by: kolaente <k@knt.li>
* Add scheduled pr merge to tables used for testing
Signed-off-by: kolaente <k@knt.li>
* Add status param to make GetPullRequestByHeadBranch reusable
Signed-off-by: kolaente <k@knt.li>
* Move "Merge now" to a seperate button to make the ui clearer
Signed-off-by: kolaente <k@knt.li>
* Update models/scheduled_pull_request_merge.go
Co-authored-by: 赵智超 <1012112796@qq.com>
* Update web_src/js/index.js
Co-authored-by: 赵智超 <1012112796@qq.com>
* Update web_src/js/index.js
Co-authored-by: 赵智超 <1012112796@qq.com>
* Re-add migration after merge
* Fix frontend lint
* Fix version compare
* Add vendored dependencies
* Add basic tets
* Make sure the api route is capable of scheduling PRs for merging
* Fix comparing version
* make vendor
* adopt refactor
* apply suggestion: User -> Doer
* init var once
* Fix Test
* Update templates/repo/issue/view_content/comments.tmpl
* adopt
* nits
* next
* code format
* lint
* use same name schema; rm CreateUnScheduledPRToAutoMergeComment
* API: can not create schedule twice
* Add TestGetBranchNamesForSha
* nits
* new go routine for each pull to merge
* Update models/pull.go
Co-authored-by: a1012112796 <1012112796@qq.com>
* Update models/scheduled_pull_request_merge.go
Co-authored-by: a1012112796 <1012112796@qq.com>
* fix & add renaming sugestions
* Update services/automerge/pull_auto_merge.go
Co-authored-by: a1012112796 <1012112796@qq.com>
* fix conflict relicts
* apply latest refactors
* fix: migration after merge
* Update models/error.go
Co-authored-by: delvh <dev.lh@web.de>
* Update options/locale/locale_en-US.ini
Co-authored-by: delvh <dev.lh@web.de>
* Update options/locale/locale_en-US.ini
Co-authored-by: delvh <dev.lh@web.de>
* adapt latest refactors
* fix test
* use more context
* skip potential edgecases
* document func usage
* GetBranchNamesForSha() -> GetRefsBySha()
* start refactoring
* ajust to new changes
* nit
* docu nit
* the great check move
* move checks for branchprotection into own package
* resolve todo now ...
* move & rename
* unexport if posible
* fix
* check if merge is allowed before merge on scheduled pull
* debugg
* wording
* improve SetDefaults & nits
* NotAllowedToMerge -> DisallowedToMerge
* fix test
* merge files
* use package "errors"
* merge files
* add string names
* other implementation for gogit
* adapt refactor
* more context for models/pull.go
* GetUserRepoPermission use context
* more ctx
* use context for loading pull head/base-repo
* more ctx
* more ctx
* models.LoadIssueCtx()
* models.LoadIssueCtx()
* Handle pull_service.Merge in one DB transaction
* add TODOs
* next
* next
* next
* more ctx
* more ctx
* Start refactoring structure of old pull code ...
* move code into new packages
* shorter names ... and finish **restructure**
* Update models/branches.go
Co-authored-by: zeripath <art27@cantab.net>
* finish UpdateProtectBranch
* more and fix
* update datum
* template: use "svg" helper
* rename prQueue 2 prPatchCheckerQueue
* handle automerge in queue
* lock pull on git&db actions ...
* lock pull on git&db actions ...
* add TODO notes
* the regex
* transaction in tests
* GetRepositoryByIDCtx
* shorter table name and lint fix
* close transaction bevore notify
* Update models/pull.go
* next
* CheckPullMergable check all branch protections!
* Update routers/web/repo/pull.go
* CheckPullMergable check all branch protections!
* Revert "PullService lock via pullID (#19520)" (for now...)
This reverts commit 6cde7c9159a5ea75a10356feb7b8c7ad4c434a9a.
* Update services/pull/check.go
* Use for a repo action one database transaction
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: delvh <dev.lh@web.de>
* Update services/issue/status.go
Co-authored-by: delvh <dev.lh@web.de>
* Update services/issue/status.go
Co-authored-by: delvh <dev.lh@web.de>
* use db.WithTx()
* gofmt
* make pr.GetDefaultMergeMessage() context aware
* make MergePullRequestForm.SetDefaults context aware
* use db.WithTx()
* pull.SetMerged only with context
* fix deadlock in `test-sqlite\#TestAPIBranchProtection`
* dont forget templates
* db.WithTx allow to set the parentCtx
* handle db transaction in service packages but not router
* issue_service.ChangeStatus just had caused another deadlock :/
it has to do something with how notification package is handled
* if we merge a pull in one database transaktion, we get a lock, because merge infoce internal api that cant handle open db sessions to the same repo
* ajust to current master
* Apply suggestions from code review
Co-authored-by: delvh <dev.lh@web.de>
* dont open db transaction in router
* make generate-swagger
* one _success less
* wording nit
* rm
* adapt
* remove not needed test files
* rm less diff & use attr in JS
* ...
* Update services/repository/files/commit.go
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* ajust db schema for PullAutoMerge
* skip broken pull refs
* more context in error messages
* remove webUI part for another pull
* remove more WebUI only parts
* API: add CancleAutoMergePR
* Apply suggestions from code review
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* fix lint
* Apply suggestions from code review
* cancle -> cancel
Co-authored-by: delvh <dev.lh@web.de>
* change queue identifyer
* fix swagger
* prevent nil issue
* fix and dont drop error
* as per @zeripath
* Update integrations/git_test.go
Co-authored-by: delvh <dev.lh@web.de>
* Update integrations/git_test.go
Co-authored-by: delvh <dev.lh@web.de>
* more declarative integration tests (dedup code)
* use assert.False/True helper
Co-authored-by: 赵智超 <1012112796@qq.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* chore: add health check endpoint
docs: update document about health check
fix: fix up Sqlite3 ping. current ping will success even if the db file is missing
fix: do not expose privacy information in output field
* refactor: remove HealthChecker struct
* Added `/api/healthz` to install routes.
This was needed for using /api/healthz endpoint in Docker healthchecks,
otherwise, Docker would never become healthy if using healthz endpoint
and users would not be able to complete the installation of Gitea.
* Update modules/cache/cache.go
* fine tune
* Remove unnecessary test code. Now there are 2 routes for installation (and maybe more in future)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Marcos de Oliveira <marcossantos@furb.br>
- dont overwrite err with nil unintentionaly
- rename CheckPRReadyToMerge to CheckPullBranchProtections
- rename prQueue to prPatchCheckerQueue
from #9307
Co-authored-by: delvh <dev.lh@web.de>
* Apply DefaultUserIsRestricted in CreateUser
* Enforce system defaults in CreateUser
Allow for overwrites with CreateUserOverwriteOptions
* Fix compilation errors
* Add "restricted" option to create user command
* Add "restricted" option to create user admin api
* Respect default setting.Service.RegisterEmailConfirm and setting.Service.RegisterManualConfirm where needed
* Revert "Respect default setting.Service.RegisterEmailConfirm and setting.Service.RegisterManualConfirm where needed"
This reverts commit ee95d3e8dc.
Targeting #14936, #15332
Adds a collaborator permissions API endpoint according to GitHub API: https://docs.github.com/en/rest/collaborators/collaborators#get-repository-permissions-for-a-user to retrieve a collaborators permissions for a specific repository.
### Checks the repository permissions of a collaborator.
`GET` `/repos/{owner}/{repo}/collaborators/{collaborator}/permission`
Possible `permission` values are `admin`, `write`, `read`, `owner`, `none`.
```json
{
"permission": "admin",
"role_name": "admin",
"user": {}
}
```
Where `permission` and `role_name` hold the same `permission` value and `user` is filled with the user API object. Only admins are allowed to use this API endpoint.
* Don't error when branch's commit doesn't exist
- If one of the branches no longer exists, don't throw an error, it's possible that the branch was destroyed during the process. Simply skip it and disregard it.
- Resolves#19541
* Don't send empty objects
* Use more minimal approach
Adds a feature [like GitHub has](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) (step 7).
If you create a new PR from a forked repo, you can select (and change later, but only if you are the PR creator/poster) the "Allow edits from maintainers" option.
Then users with write access to the base branch get more permissions on this branch:
* use the update pull request button
* push directly from the command line (`git push`)
* edit/delete/upload files via web UI
* use related API endpoints
You can't merge PRs to this branch with this enabled, you'll need "full" code write permissions.
This feature has a pretty big impact on the permission system. I might forgot changing some things or didn't find security vulnerabilities. In this case, please leave a review or comment on this PR.
Closes#17728
Co-authored-by: 6543 <6543@obermui.de>
This gets the necessary data to the issuelist for it to support a clickable commit status icon which pops up the full list of commit statuses related to the commit. It accomplishes this without any additional queries or fetching as the existing codepath was already doing the necessary work but only returning the "last" status. All methods were wrapped to call the least-filtered version of each function in order to maximize code reuse.
Note that I originally left `getLastCommitStatus()` in `pull.go` which called to the new function, but `make lint` complained that it was unused, so I removed it. I would have preferred to keep it, but alas.
The only thing I'd still like to do here is force these popups to happen to the right by default instead of the left. I see that the only other place this is popping up right is on view_list.tmpl, but I can't figure out how/why right now.
Fixes#18810
* Improve dashboard's repo list performance
- Avoid a lot of database lookups for all the repo's, by adding a
undocumented "minimal" mode for this specific task, which returns the
data that's only needed by this list which doesn't require any database
lookups.
- Makes fetching these list faster.
- Less CPU overhead when a user visits home page.
* Refactor javascript code + fix Fork icon
- Use async in the function so we can use `await`.
- Remove `archivedFilter` check for count, as it doesn't make sense to
show the count of repos when you can't even see them(as they are
filited away).
* Add `count_only`
* Remove uncessary code
* Improve comment
Co-authored-by: delvh <dev.lh@web.de>
* Update web_src/js/components/DashboardRepoList.js
Co-authored-by: delvh <dev.lh@web.de>
* Update web_src/js/components/DashboardRepoList.js
Co-authored-by: delvh <dev.lh@web.de>
* By default apply minimal mode
* Remove `minimal` paramater
* Refactor count header
* Simplify init
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
- Don't panic on `ErrEmailInvalid`, this was caused due that we were
trying to force `ErrEmailCharIsNotSupported` interface, which panics.
- Resolves#19397
When a mirror repo interval is updated by the UI it is rescheduled with that interval
however the API does not do this. The API also lacks the enable_prune option.
This PR adds this functionality in to the API Edit Repo endpoint.
Signed-off-by: Andrew Thornton <art27@cantab.net>
- Add reqSignIn to `/user/task/{task}` as it specific to a logged in user currently not-logged in user could cause a NPE.
- Move `/user/stopwatch` & `/user/search` middleware before the actual function is called, because functions are executed in order and currently hadn't any effect and could as well cause a NPE due to that.
- Remove `/user/active` reqSignIn middleware, because when you want to active a account you're not "signed in" so it doesn't make sense to add that middleware.
Do a refactoring to the CSRF related code, remove most unnecessary functions.
Parse the generated token's issue time, regenerate the token every a few minutes.
Reusing `/api/v1` from Gitea UI Pages have pros and cons.
Pros:
1) Less code copy
Cons:
1) API/v1 have to support shared session with page requests.
2) You need to consider for each other when you want to change something about api/v1 or page.
This PR moves all dependencies to API/v1 from UI Pages.
Partially replace #16052
Remove two unmaintained vendor packages `i18n` and `paginater`. Changes:
* Rewrite `i18n` package with a more clear fallback mechanism. Fix an unstable `Tr` behavior, add more tests.
* Refactor the legacy `Paginater` to `Paginator`, test cases are kept unchanged.
Trivial enhancement (no breaking for end users):
* Use the first locale in LANGS setting option as the default, add a log to prevent from surprising users.
* An attempt to sync a non-mirror repo must give 400 (Bad Request)
* add missing return statement
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
The main purpose is to refactor the legacy `unknwon/com` package.
1. Remove most imports of `unknwon/com`, only `util/legacy.go` imports the legacy `unknwon/com`
2. Use golangci's depguard to process denied packages
3. Fix some incorrect values in golangci.yml, eg, the version should be quoted string `"1.18"`
4. Use correctly escaped content for `go-import` and `go-source` meta tags
5. Refactor `com.Expand` to our stable (and the same fast) `vars.Expand`, our `vars.Expand` can still return partially rendered content even if the template is not good (eg: key mistach).
Skip `checkAppUrl` message on installation page because the ROOT_URL is not determined yet
Move global var `supportedDbTypeNames` into `install.Init` as a local var
Follows #19266, #8553, Close#18553, now there are only three `Run..(&RunOpts{})` functions.
* before: `stdout, err := RunInDir(path)`
* now: `stdout, _, err := RunStdString(&git.RunOpts{Dir:path})`
Continues on from #19202.
Following the addition of pprof labels we can now more easily understand the relationship between a goroutine and the requests that spawn them.
This PR takes advantage of the labels and adds a few others, then provides a mechanism for the monitoring page to query the pprof goroutine profile.
The binary profile that results from this profile is immediately piped in to the google library for parsing this and then stack traces are formed for the goroutines.
If the goroutine is within a context or has been created from a goroutine within a process context it will acquire the process description labels for that process.
The goroutines are mapped with there associate pids and any that do not have an associated pid are placed in a group at the bottom as unbound.
In this way we should be able to more easily examine goroutines that have been stuck.
A manager command `gitea manager processes` is also provided that can export the processes (with or without stacktraces) to the command line.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This make checks in one single place so they dont differ and maintainer can not forget a check in one place while adding it to the other .... ( as it's atm )
Fix:
* The API does ignore issue dependencies where Web does not
* The API checks if "IsSignedIfRequired" where Web does not - UI probably do but nothing will some to craft custom requests
* Default merge message is crafted a bit different between API and Web if not set on specific cases ...
This follows
* https://github.com/go-gitea/gitea/issues/18553
Introduce `RunWithContextString` and `RunWithContextBytes` to help the refactoring. Add related unit tests. They keep the same behavior to save stderr into err.Error() as `RunInXxx` before.
Remove `RunInDirTimeoutPipeline` `RunInDirTimeoutFullPipeline` `RunInDirTimeout` `RunInDirTimeoutEnv` `RunInDirPipeline` `RunInDirFullPipeline` `RunTimeout`, `RunInDirTimeoutEnvPipeline`, `RunInDirTimeoutEnvFullPipeline`, `RunInDirTimeoutEnvFullPipelineFunc`.
Then remaining `RunInDir` `RunInDirBytes` `RunInDirWithEnv` can be easily refactored in next PR with a simple search & replace:
* before: `stdout, err := RunInDir(path)`
* next: `stdout, _, err := RunWithContextString(&git.RunContext{Dir:path})`
Other changes:
1. When `timeout <= 0`, use default. Because `timeout==0` is meaningless and could cause bugs. And now many functions becomes more simple, eg: `GitGcRepos` 9 lines to 1 line. `Fsck` 6 lines to 1 line.
2. Only set defaultCommandExecutionTimeout when the option `setting.Git.Timeout.Default > 0`
The last PR about clone buttons introduced an JS error when visiting an empty repo page:
* https://github.com/go-gitea/gitea/pull/19028
* `Uncaught ReferenceError: isSSH is not defined`, because the variables are scoped and doesn't share between sub templates.
This:
1. Simplify `templates/repo/clone_buttons.tmpl` and make code clear
2. Move most JS code into `initRepoCloneLink`
3. Remove unused `CloneLink.Git`
4. Remove `ctx.Data["DisableSSH"] / ctx.Data["ExposeAnonSSH"] / ctx.Data["DisableHTTP"]`, and only set them when is is needed (eg: deploy keys / ssh keys)
5. Introduce `Data["CloneButton*"]` to provide data for clone buttons and links
6. Introduce `Data["RepoCloneLink"]` for the repo clone link (not the wiki)
7. Remove most `ctx.Data["PageIsWiki"]` because it has been set in the `/wiki` middleware
8. Remove incorrect `quickstart` class in `migrating.tmpl`
This PR adds the necessary work to make it possible to create files on empty
repos using the API.
Fix#10993
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove the global methods but create dynamiclly
* Fix lint
* Fix windows lint
* Fix windows lint
* some improvements
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
There is a bug in the system webhooks whereby the active state is not checked when
webhooks are prepared and there is a bug that deactivating webhooks do not prevent
queued deliveries.
* Only add SystemWebhooks to the prepareWebhooks list if they are active
* At the time of delivery if the underlying webhook is not active mark it
as "delivered" but with a failed delivery so it does not get delivered.
Fix#19220
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Set the default branch for repositories generated from templates
* Allows default branch to be set through the API for repos generated from templates
* Update swagger API template
* Only set default branch to the one from the template if not specified
* Use specified default branch if it exists while generating git commits
Fix#19082
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
This PR adds a middleware which sets a ContextUser (like GetUserByParams before) in a single place which can be used by other methods. For routes which represent a repo or org the respective middlewares set the field too.
Also fix a bug in modules/context/org.go during refactoring.
- Make a restriction on which issues can be shown based on if you the user or team has write permission to the repository.
- Fixes a issue whereby you wouldn't see any associated issues with a specific team on a organization if you wasn't a member(fixed by zeroing the User{ID} in the options).
- Resolves#18913
When a new branch is pushed the old SHA is always listed as the empty sha and thus the compare link that is created does not work correctly.
Therefore when creating the compare link for new branches:
1. Attempt to get the parent of the first commit and use that as the basis
for the compare link.
2. If this is not possible make a comparison to the default branch
3. Finally if that is not possible simply do not show a compare link.
However, there are multiple broken compare links remaining therefore, in order for these to not break we will simply make the compare link redirect to the default branch.
Fix#19144
Signed-off-by: a1012112796 <1012112796@qq.com>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* Clean paths when looking in Storage
Ensure paths are clean for minio aswell as local storage.
Use url.Path not RequestURI/EscapedPath in storageHandler.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Apply suggestions from code review
Co-authored-by: Lauris BH <lauris@nix.lv>
Extend issues/new/choose to pass the project id and change New Issue
link on project page to use new/choose
Fix#19170
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Remove `db.DefaultContext` usage in routers, use `ctx` directly
* Use `ctx` directly if there is one, remove some `db.DefaultContext` in `services`
* Use ctx instead of db.DefaultContext for `cmd` and some `modules` packages
* fix incorrect context usage
