- Add a new script that proccess the localization files and verify that
they only contain HTML according to our strictly defined rules.
- This should make adding malicious HTML near-impossible.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5703
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Since https://github.com/go-gitea/gitea/pull/25686, a few `interface{}`
have sneaked into the codebase. Add this replacement to `make fmt` to
prevent this from happening again.
Ideally a linter would do this, but I haven't found any suitable.
(cherry picked from commit c77e8140bc2ac6521dbebfb77613dce2648bfcb8)
Conflicts:
- .gitattributes
Trivial conflict resolved by picking our choice of language
for `*.tmpl` files.
Instead of failing or do things that could be damaging, this script
prints a deprecation notice. It fixes the unnecessary breaking
change introduced in `[I18n] tooling and process`.
https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/version-management/forgejo/default.nix#L80
$ make merge-locales
NOT NEEDED: THIS IS A NOOP AS OF Forgejo 7.0 BUT KEPT FOR BACKWARD COMPATIBILITY
$ go run build/merge-forgejo-locales.go
NOT NEEDED: THIS IS A NOOP AS OF Forgejo 7.0 BUT KEPT FOR BACKWARD COMPATIBILITY
Also remove the build/crowdin-to-weblate.sh script that was never
needed.
- Move some scripts from `build` to new `tools` dir. Eventually i would
like to move all but let's do it step-by-step.
- Add dir to eslint and move the files into vars.
- Update docs accordingly.
- While updating docs I noticed we were incorrectly having `public/img`
path still in a few places. Replace those with the current
`public/assets/img`.
---------
Co-authored-by: Nanguan Lin <nanguanlin6@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit 095fdd691dd1a7d7748372cc73e7708278c80933)
This plugin has a few useful rules. The only thing I dislike about it is
that it pulls in a rather big number of dependencies for react-related
rules we don't use, but it can't really be avoided.
Rule docs:
https://github.com/github/eslint-plugin-github?tab=readme-ov-file#rules
(cherry picked from commit 26b17537e651fe93ef9b64f961633cb4c0b8c2c3)
Before:
* Translations were imported from Crowdin via Gitea
* New strings were added by Forgejo in feature branches but never translated
* The build/merge-forgejo-locales.go script in the forgejo-i18n branch
added/replaced Forgejo specific strings in the translated files at
build time
* Conflicts arising from Gitea changes in locale_en-US.ini were resolved by moving around
Forgejo strings
Transition:
* The build/merge-forgejo-locales.go is run once on the source
* The result is commited to the forgejo-development branch
After:
* Translation are exported and imported from translate.codeberg.org
* build/crowdin-to-weblate.sh run is run when rebasing (soft-fork)
or cherry-picking (hard-fork) to collect new translations from
Crowdin via Gitea
* In case of a conflict Forgejo translations are authoritative
* In the absence of a conflict Crowdin translations are uploaded to Weblate
(cherry picked from commit 9b5b714048)
(cherry picked from commit 6c7b5e322d)
Move `public/*` to `public/assets/*`
Some old PRs (like #15219) introduced inconsistent directory system.
For example: why the local directory "public" is accessed by
`http://site/assets`? How to serve the ".well-known" files properly in
the public directory?
For convention rules, the "public" directory is widely used for the
website's root directory. It shouldn't be an exception for Gitea.
So, this PR makes the things consistent:
* `http://site/assets/foo` means `{CustomPath}/public/assets/foo`.
* `{CustomPath}/public/.well-known` and `{CustomPath}/public/robots.txt`
can be used in the future.
This PR is also a prerequisite for a clear solution for:
* #21942
* #25892
* discourse.gitea.io: [.well-known path serving custom files behind
proxy?](https://discourse.gitea.io/t/well-known-path-serving-custom-files-behind-proxy/5445/1)
This PR is breaking for users who have custom "public" files (CSS/JS).
After getting approvals, I will update the documents.
----
## ⚠️ BREAKING ⚠️
If you have files in your "custom/public/" folder, please move them to
"custom/public/assets/".
---------
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Giteabot <teabot@gitea.io>
The INI package has many bugs and quirks, and in fact it is
unmaintained.
This PR is the first step for the INI package refactoring:
* Use Gitea's "config_provider" to provide INI access
* Deprecate the INI package by golangci.yml rule
Replace #22117. Implement it in a more maintainable way.
Some licenses have placeholders e.g. the BSD licenses start with this
line:
```
Copyright (c) <year> <owner>.
```
This PR replaces the placeholders with the correct value when initialize
a new repo.
### FAQ
- Why not use a regex?
It will be a pretty complicated regex which could be hard to maintain.
- There're still missing placeholders.
There are over 500 licenses, it's impossible for anyone to inspect all
of them alone. Please help to add them if you find any, and it is also
OK to leave them for the future.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
This will ensure that the file always has a final newline. I'm not sure
where this bug with inconsistent final newline actually comes from, it
is likely Windows-related.
---------
Co-authored-by: delvh <dev.lh@web.de>
None of the features of `unrolled/render` package is used.
The Golang builtin "html/template" just works well. Then we can improve
our HTML render to resolve the "$.root.locale.Tr" problem as much as
possible.
Next step: we can have a template render pool (by Clone), then we can
inject global functions with dynamic context to every `Execute` calls.
Then we can use `{{Locale.Tr ....}}` directly in all templates , no need
to pass the `$.root.locale` again and again.
Follow #23633 and #23240Close#23814
Now we almost have a complete test set for Gitea's LocalStore.
This PR is still a quick fix for the legacy locale system (see the
TODOs), to resolve the problems fundamentally, it needs more work in the
future.
ps: there are more broken translation strings in 1.20 (main), most of
them are still caused by incorrect quoting/unquoting. For example,
translators might write text ``` `my text ```, such incorrect encoding
might break crowdin & Gitea's locale package.
In the future, a Go `update-locales.go` should replace the legacy
`update-locales.sh`.
# Why this PR comes
At first, I'd like to help users like #23636 (there are a lot)
The unclear "Internal Server Error" is quite anonying, scare users,
frustrate contributors, nobody knows what happens.
So, it's always good to provide meaningful messages to end users (of
course, do not leak sensitive information).
When I started working on the "response message to end users", I found
that the related code has a lot of technical debt. A lot of copy&paste
code, unclear fields and usages.
So I think it's good to make everything clear.
# Tech Backgrounds
Gitea has many sub-commands, some are used by admins, some are used by
SSH servers or Git Hooks. Many sub-commands use "internal API" to
communicate with Gitea web server.
Before, Gitea server always use `StatusCode + Json "err" field` to
return messages.
* The CLI sub-commands: they expect to show all error related messages
to site admin
* The Serv/Hook sub-commands (for git clients): they could only show
safe messages to end users, the error log could only be recorded by
"SSHLog" to Gitea web server.
In the old design, it assumes that:
* If the StatusCode is 500 (in some functions), then the "err" field is
error log, shouldn't be exposed to git client.
* If the StatusCode is 40x, then the "err" field could be exposed. And
some functions always read the "err" no matter what the StatusCode is.
The old code is not strict, and it's difficult to distinguish the
messages clearly and then output them correctly.
# This PR
To help to remove duplicate code and make everything clear, this PR
introduces `ResponseExtra` and `requestJSONResp`.
* `ResponseExtra` is a struct which contains "extra" information of a
internal API response, including StatusCode, UserMsg, Error
* `requestJSONResp` is a generic function which can be used for all
cases to help to simplify the calls.
* Remove all `map["err"]`, always use `private.Response{Err}` to
construct error messages.
* User messages and error messages are separated clearly, the `fail` and
`handleCliResponseExtra` will output correct messages.
* Replace all `Internal Server Error` messages with meaningful (still
safe) messages.
This PR saves more than 300 lines, while makes the git client messages
more clear.
Many gitea-serv/git-hook related essential functions are covered by
tests.
---------
Co-authored-by: delvh <dev.lh@web.de>
* Clean the "tools" directory. The "tools" directory contains only two
files, move them.
* The "external_renderer.go" works like "cat" command to echo Stdin to
Stdout , to help testing.
* The `// gobuild: external_renderer` is incorrect, there should be no
space: `//gobuild: external_renderer`
* The `fmt.Print(os.Args[1])` is not a well-defined behavior, and it's
never used.
* The "watch.sh" is for "make watch", it's somewhat related to "build"
* After this PR, there is no "tools" directory, the project root
directory looks slightly simpler than before.
* Remove the legacy "contrib/autoboot.sh", there is no
"gogs_supervisord.sh"
* Remove the legacy "contrib/mysql.sql", it's never mentioned anywhere.
* Remove the legacy "contrib/pr/checkout.go", it has been broken for
long time, and it introduces unnecessary dependencies of the main code
base.
It use old en-US locales as reference, fill the old other locales with
new locales.
----
## More broken translations
Many translations are still broken. The reason is: at the last time
restoring the ini to crowdin, many semicolon are treated as comments.
Two kinds of broken strings:
### Some translations can be re-translated
<details>
```
skipping options/locale/locale_si-LK.ini org teams.add_nonexistent_repo
skipping options/locale/locale_tr-TR.ini repo commits.search.tooltip
skipping options/locale/locale_es-ES.ini repo settings.trust_model.committer.desc
skipping options/locale/locale_es-ES.ini admin dashboard.new_version_hint
skipping options/locale/locale_pt-PT.ini org teams.add_nonexistent_repo
skipping options/locale/locale_hu-HU.ini install require_sign_in_view_popup
skipping options/locale/locale_hu-HU.ini repo migrate.invalid_local_path
skipping options/locale/locale_id-ID.ini repo migrate.invalid_local_path
skipping options/locale/locale_id-ID.ini org teams.add_nonexistent_repo
skipping options/locale/locale_de-DE.ini repo settings.protect_protected_file_patterns_desc
```
</details>
So this PR also does some small changes on them, to trigger the
re-translation.
### The `locale_el-GR.ini` contains many broken tranlsations
I guess we should reset them from crowdin side, then translators can
re-translate them.
----
Update: in latest main, the strings have been fixed.
## TODO
Update: the el-GR translators have done great job and fixes these broken
translations.
<details>
Merge this PR ASAP and upload `locale_el-GR.ini` to crowdin to remove
broken strings.
Out-dated, fixed in main.
</details>
---------
Co-authored-by: delvh <dev.lh@web.de>
- ~~Make scripts work from any directory~~
- Detect sed version just like Makefile does
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
The locales of Gitea has been broken for long time, till now, it's still
not fully fixed.
One of the root problems is that the `ini` library is quite quirky and
the `update-locales` script doesn't work well for all cases.
This PR fixes the `update-locales` script to make it satisfy `ini`
library and the crowdin.
See the comments for more details.
The `locale_zh-CN.ini` is an example, it comes from crowdin and is
processed by the new `update-locales.sh`. Especially see the `feed_of`:
https://github.com/go-gitea/gitea/pull/23240/files#diff-321f6ca4eae1096eba230e93c4740f9903708afe8d79cf2e57f4299786c4528bR268
This PR just consumes the
[hcaptcha](https://gitea.com/jolheiser/hcaptcha) and
[haveibeenpwned](https://gitea.com/jolheiser/pwn) modules directly into
Gitea.
Also let this serve as a notice that I'm fine with transferring my
license (which was already MIT) from my own name to "The Gitea Authors".
Signed-off-by: jolheiser <john.olheiser@gmail.com>
As suggest by Go developers, use `filepath.WalkDir` instead of
`filepath.Walk` because [*Walk is less efficient than WalkDir,
introduced in Go 1.16, which avoids calling `os.Lstat` on every file or
directory visited](https://pkg.go.dev/path/filepath#Walk).
This proposition address that, in a similar way as
https://github.com/go-gitea/gitea/pull/22392 did.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Gitea emoji dataset was out of date because it gets manually built and
hasn't been rebuilt since it was added. This means Gitea doesn't
recognize some newer emoji or changes to existing ones.
After changing the max unicode version to 14 I just ran: `go run
build/generate-emoji.go`
This should address the initial issue seen in #22153 where Gitea doesn't
recognize a standard alias used elsewhere when importing content.
14 is the latest supported version from the upstream source as 15 is not
widely supported (in their opinion) yet
- Update all JS dependencies to latest version
- Enable unicorn/prefer-node-protocol and autofix issues
- Regenerate SVGs
- Add some comments to eslint rules
- Tested build, Mermaid and Katex rendering
Change all license headers to comply with REUSE specification.
Fix#16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
`fmt-check` now simply does `fmt` before and relies on `git diff` like
other checks like 'tidy-check' already do, so we can remove the argument
in the tool that handles printing changed files.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
We should make sure we're using the same version across the codebase.
* We upgraded in #19918 but forgot about the following line
6bb6a108e0/build/code-batch-process.go (L273)
Signed-off-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
We can not have the `frontend` target depend on golang because of they
way drone is set up. Move the `go-licenses` generation back into `tidy`
where it will now also be checked for consistency during `tidy-check`.
(I assume all `main` branch builds should currently fail [like
this](https://drone.gitea.io/go-gitea/gitea/60244/1/11)).
The reasony why it shouldn't be treated the same as for example `go
generate` is because output files are checked in. tidy is imho the
optimal target to run this after.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This removes the JS dependency in the checks pipeline. JSON output is
different because the previous JS did indent the license data
differently and a JSON key was changed, but the end result is the same
as it gets re-indented by wepack.
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
`make go-licenses` will generate `assets/go-licenses.json` which is then included in the webpack build.
This step depends on both go and node being present, so unfortunately, I could not automate the generation by hooking it up to `tidy` as that target is triggered on CI where we do not have a docker image with both go an node.
It should be ran from time to time, ideally after each go mod update.
- Remove __dirname, use file URLs instead
- Upgrade fabric dependency
- Use fs/promises syntax, this breaks node 12 but we require 14 already
The change in public/img/favicon.svg is not caused by the fabric
upgrade, but it seems it was not properly generated when introduced.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Changed the filename of the favicon SVG
This allows the user to have a favicon which differs from the logo.
* Added favicon.svg
This is needed to accommodate the changes for allowing the user to have a differing logo and favicon
* Adjusted page to accommodate what icon is used as favicon
* Added functionality to also generate the favicon.svg via generate-images.js
* Adjusted the description for the new favicon compatibility
Co-authored-by: silverwind <me@silverwind.io>
* Updated generate-images.js to generate favicons from a separate favicons.svg file
This belongs to PR #18542.
* Added description on how custom favicons can be generated
* Replaced space indents with tabs
* Synced changes with current state of the file
* Synced changes with current state of the file
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
