mirror/forgejo
1
0
Fork 0
mirror of https://codeberg.org/forgejo/forgejo.git synced 2024-11-28 20:26:19 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

Merge branch 'master' into feature/restricted-users

Browse source
This commit is contained in:
Antoine GIRARD 2020-01-08 12:31:28 +01:00 committed by GitHub
commit 1f7b0fc4f4
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
97 changed files with 4427 additions and 1267 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
.changelog.yml
View file

@ -5,7 +5,7 @@ groups:
labels:
- kind/breaking
-
name: FEATURE
name: FEATURES
labels:
- kind/feature
-
@ -13,7 +13,7 @@ groups:
labels:
- kind/bug
-
name: ENHANCEMENT
name: ENHANCEMENTS
labels:
- kind/enhancement
- kind/refactor
@ -41,4 +41,4 @@ groups:
- kind/docs
-
name: MISC
default: true
default: true

432
CHANGELOG.md
View file

@ -4,6 +4,386 @@ This changelog goes through all the changes that have been made in each release
without substantial changes to our git log; to see the highlights of what has
been added to each release, please refer to the [blog](https://blog.gitea.io).
## [1.11.0-RC1](https://github.com/go-gitea/gitea/releases/tag/v1.11.0-rc1) - 2020-01-07
* BREAKING
* Remove unused endpoints (#9538)
* Prefix all user-generated IDs in markup (#9477)
* Enforce Gitea environment for pushes (#8982)
* Hide some user information via API if user have no enough permission (#8655)
* Move startpage/homepage translation to crowdin (#8596)
* FEATURES
* Webhooks should only show sender if it makes sense (#9601)
* Provide Default messages for merges (#9393)
* Add description to labels on create issue (#9392)
* Graceful Queues: Issue Indexing and Tasks (#9363)
* Default NO_REPLY_ADDRESS to DOMAIN (#9325)
* Allow FCGI over unix sockets (#9298)
* Graceful: Xorm, RepoIndexer, Cron and Others (#9282)
* Add API for Reactions (#9220)
* Graceful: Cancel Process on monitor pages & HammerTime (#9213)
* Graceful: Allow graceful restart for unix sockets (#9113)
* Graceful: Allow graceful restart for fcgi (#9112)
* Sign protected branches (#8993)
* Add Graceful shutdown for Windows and hooks for shutdown of goroutines (#8964)
* Add Gitea icon to Emojis (#8950)
* Expand/Collapse Files and Blob Excerpt while Reviewing/Comparing code (#8924)
* Allow Custom Reactions (#8886)
* Close/reopen issues by keywords in titles and comments (#8866)
* Allow incompletely specified Time Formats (#8816)
* Prevent upload (overwrite) of lfs locked file (#8769)
* Template Repositories (#8768)
* Add /milestones endpoint (#8733)
* Make repository management section handle lfs locks (#8726)
* Respect LFS File Lock on UI (#8719)
* Add team option to grant rights for all organization repositories (#8688)
* Enabling and disabling the commit button to prevent empty commits (web editor) (#8590)
* Add setting to disable BASIC authentication (#8586)
* Expose db.SetMaxOpenConns and allow non MySQL dbs to set conn pool params (#8528)
* Allow Protected Branches to Whitelist Deploy Keys (#8483)
* Push to create repo (#8419)
* Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631)
* Add basic repository lfs management (#7199)
* BUGFIXES
* Disable remove button on repository teams when have access to all (#9640)
* Clean up old references on branch delete (#9614)
* Hide public repos owned by private orgs (#9609)
* Fix access issues on milestone and issue overview pages. (#9603)
* Fix error logged when repos qs is empty (#9591)
* Dont trigger notification twice on issue assignee change (#9582)
* Fix mirror pushed commit actions (#9572)
* Allow only specific columns to be updated on issue via API (#9189) (#9539)
* Fix default avatar for ghost user (#9536)
* Fix download of release attachments with same name (#9529)
* Resolve deprecated INI conversion (#9525)
* Ignore empty avatars during database migration (#9520)
* Fix deleted branch isn't removed when push the branch again (#9516)
* Fix repository issues pagination bug when there are more than one label filter (#9512)
* Fix SetExpr failed (#9506)
* Remove obsolete file private/push_update.go (#9503)
* When recreating hooks, delete them first so they are recreated with the umask (#9502)
* Properly enforce gitea environment for pushes (#9501)
* Fix datarace on repo indexer queue (#9490)
* Add call to load repo prior to redirect in add/remove dependency code (#9484)
* Wrap the code indexer (#9476)
* Use Req.URL.RequestURI() to cope with FCGI urls (#9473)
* Set default ssh.minimum_key_sizes (#9466)
* Fixed issue with paging in /repos/{owner}/{repo}/git/trees/{sha} api (#9459)
* Fix wrong notification on merge (#9450)
* Issue with Migration rule v111 (#9449)
* Trigger webhook when deleting a branch after merging a PR (#9424)
* Add migration to sanitize repository original_url (#9423)
* Use OriginalURL instead of CloneAddr in migration logging (#9418)
* Push update after branch is restored (#9416)
* Fix wrong migration (#9381)
* Fix show repositories filter (#9234) (#9379)
* Fix Slack webhook payload title generation to work with Mattermost (#9378)
* Fix double webhook for new PR (#9375)
* AuthorizedKeysCommand should not query db directly (#9371)
* Fix missed change to GetManager() (#9361)
* Fix cache problem on dashboard (#9358)
* RepoIndexer: DefaultBranch needs to be prefixed by BranchPrefix (#9356)
* Fix protected branch using IssueID (#9348)
* Fix nondeterministic behavior (#9341)
* Fix PR/issue redirects when having external tracker (#9339)
* Remove release attachments which repository has been deleted (#9334)
* Fix issue indexer not triggered when migrating a repository (#9332)
* Add SyncTags to uploader interface (#9326)
* Fix bug that release attachment files not deleted when deleting repository (#9322)
* Only sync tags after all migration release batches are completed (#9319)
* File Edit: Author/Committer interchanged (#9297)
* prebuild CSS/JS before xgo release binaries (#9293)
* Log: Ensure FLAGS=none shows no flags (#9287)
* Make Diff Detail on Pull Request Changed File UI always on Top (#9280)
* Switch CSS minifier to cssnano (#9260)
* Fix latest docker image haven't include static files. (#9252)
* Don't link wiki revision to commit (#9244)
* Change review content column to type text in db (#9229)
* Fixed topic regex pattern and added search by topic links after save (#9219)
* Add language to user API responce (#9215)
* Correct tooltip message blocked by dependencies (#9211)
* Add SimpleMDE and Fix Image Paste for Issue/Comment Editor (#9197)
* Fix panic when diff (#9187)
* Fix #9151 - smtp logger configuration sendTos should be an array (#9154)
* Fix max length check and limit in multiple repo forms (#9148)
* Always Show Password Field on Link Account Sign-in Page (#9147)
* Properly fix displaying virtual session provider in admin panel (#9137)
* Fix race condition on indexer (#9136)
* Fix team links in HTML rendering (#9127)
* Fix race condition in ReplaceSanitizer (#9123)
* Fix what information is shown about user in API (#9115)
* Fix nil context user for template repositories (#9099)
* Hide given credentials for migrated repos. (#9097)
* Fix reCAPTCHA API URL (#9083)
* Fix password checks on admin create/edit user (#9076)
* Update golang.org/x/crypto vendor to use acme v2 (#9056)
* Ensure Written is set in GZIP ProxyResponseWriter (#9018)
* Fix wrong system notice when repository is empty (#9010)
* Fix broken link to branch from issue list (#9003)
* Fix bug when pack js (#8992)
* New review approvals shouldn't require a message (#8991)
* Shadow password correctly for session config (#8984)
* Don't send notification on pending reviews (#8943)
* Fix Notify Create Ref Error on tag creation (#8936)
* Convert EOL to UNIX-style to render MD properly (#8925)
* Migrate temp_repo.go to use git.NewCommand (#8918)
* Fix issue with user.fullname (#8902)
* Add Close() method to gogitRepository (#8901)
* Enable punctuations ending mentions (#8889)
* Fix password complexity check on registration (#8887)
* Fix require external registration password (#8885)
* Fix edit content button on migrated issue content (#8877)
* Fix permission checks for close/reopen from commit (#8875)
* Fix API Bug (fail on empty assignees) (#8873)
* Stop using git count-objects and use raw directory size for repository (#8848)
* Fix count for commit graph last page (#8843)
* Fix to close opened io resources as soon as not needed (#8839)
* Improve notification (#8835)
* Fix new user form for non-local users (#8826)
* Fix: remove duplicated signed commit icons (#8820)
* Fix (open/closed) issue count when label excluded (#8815)
* Fix SSH2 conditional in key parsing code (#8806)
* Fix 500 when edit hook (#8782)
* On windows set core.longpaths true (#8776)
* Fix commit expand button to not go to commit link (#8745)
* Avoid re-issuing redundant cross-references. (#8734)
* Fix milestone close timestamp function (#8728)
* Move webhook codes from service to webhook notification (#8712)
* Show zero lines on the line counter if the file empty (#8700)
* Fix deadline on update issue or PR via API (#8696)
* make call createMilestoneComment on newIssue func (#8678)
* Send tag create and push webhook when release created on UI (#8671)
* Prevent chrome download page as html with alt + click (#8669)
* Fix 500 when getting user as unauthenticated user (#8653)
* Graceful fixes (#8645)
* Add SubURL to redirect path (#8632) (#8634)
* Fix extra columns from `label` table (#8633)
* Add SubURL to redirect path for transferred/renamed repos (#8632)
* Fix bug when migrate from API (#8631)
* Allow to merge if file path contains " or \ (#8629)
* Prevent removal of non-empty emoji panel following selection of duplicate (#8609)
* Ensure default gpg settings not nil and found commits have reference to repo (#8604)
* Set webhook Content-Type for application/x-www-form-urlencoded (#8599)
* Fix #8582 by handling empty repos (#8587)
* Fix of the diff statistics view on pull request's (#8581)
* Fix bug on pull requests when transfer head repository (#8564)
* Fix template error on account page (#8562)
* Allow externalID to be UUID (#8551)
* Fix ignored error on editorconfig api (#8550)
* Fix user avatar name (#8547)
* Ensure that GitRepo is set on Empty repositories (#8539)
* Add missed close in ServeBlobLFS (#8527)
* Fix migrate mirror 500 bug (#8526)
* Fix password complexity regex for special characters (on master) (#8525)
* ENHANCEMENTS
* Add a /user/login landing page option (#9622)
* Some more e-mail notification fixes (#9596)
* Add branch protection option to block merge on requested changes. (#9592)
* Add footer extra links template (#9576)
* Fix for a wrong URL in activity page of repository. (#9571)
* Update default issue template (#9568)
* Change markdown rendering from blackfriday to goldmark (#9533)
* Extend file create api with dates (#9464)
* Add ActionCommentPull action (#9456)
* Response for context on retry database connection (#9444)
* Refactor webhooks to reduce code duplication (#9422)
* update couchbase deps for new license (#9419)
* Add .ignore file for search tools (#9417)
* Remove unsued struct (#9405)
* Hide not allowed Reactions (#9387)
* Remove text from action-only webhooks (#9377)
* Move PushToBaseRepo from models to services/pull (#9352)
* Site admin could view org's members (#9346)
* Sleep longer if request speed is over github limitation (#9335)
* Refactor comment (#9330)
* Refactor code indexer (#9313)
* Remove SavePatch and generate patches on the fly (#9302)
* Move some pull request functions from models to services (#9266)
* Update JS dependencies (#9255)
* Show label list on label set (#9251)
* Redirect issue if repo has configured external tracker. (#9247)
* Allow kbd tags (#9245)
* Remove unused comment actions (#9222)
* Fixed errors logging in dump.go (#9218)
* Expose release counter to repo API response (#9214)
* Make consistent links to repository in the Slack/Mattermost notificiations (#9205)
* Expose pull request counter to repo API response (#9202)
* Extend TrackedTimes API (#9200)
* Extend StopWatch API (#9196)
* Move code indexer related code to a new package (#9191)
* Docker: ask s6 to stop all service when gitea stop (#9171)
* Variable expansion in repository templates (#9163)
* Add avatar and issue labels to template repositories (#9149)
* Show single review comments in the PR conversation tab (#9143)
* Extract createComment (#9125)
* Move PushUpdateOptions from models to repofiles (#9124)
* Alternate syntax for cross references (#9116)
* Add USE_SERVICE_WORKER setting (#9110)
* Only show part of members on orgnization dashboard and add paging for orgnization members page (#9092)
* Explore page: Add topic param to pagination (#9077) (#9078)
* Markdown: Sanitizier Configuration (#9075)
* Add password requirement info on error (#9074)
* Allow authors to use act keywords in PR content (#9059)
* Move modules/gzip to gitea.com/macaron/gzip (#9058)
* Branch protection: Possibility to not use whitelist but allow anyone with write access (#9055)
* Context menus for comments, add quote reply (#9043)
* Update branch API endpoint to show effective branch protection. (#9031)
* Move git graph from models to modules/graph (#9027)
* Move merge actions to notification (#9024)
* Move mirror sync actions to notification (#9022)
* Add retry for migration http/https requests (#9019)
* Rewrite delivery of issue and comment mails (#9009)
* Add review comments to mail notifications (#8996)
* Refactor pull request review (#8954)
* Githook highlighter (#8932)
* Add git hooks and webhooks to template repositories; move to services (#8926)
* Only view branch or tag if it match refType requested. (#8899)
* Drop Admin attribute based on LDAP when login (continue #1743) (#8849)
* Add additional periods to activity page (#8829)
* Update go-org to optimize code (#8824)
* Move some actions to notification/action (#8779)
* Webhook support custom proxy (#8760)
* Fix API deadline removal (#8759)
* Mark review comment as invalidated when file is deleted (#8751)
* Move pull list code to a separate file (#8748)
* Move webhook to a standalone package under modules (#8747)
* Multi repo select on issue page (#8741)
* apply exclude label on milestone issue list (#8739)
* Move issue notifications and assignee man (#8713)
* Move issue change content from models to service (#8711)
* Move issue change status from models to service (#8691)
* Move more issue assignee code from models to issue service (#8690)
* Create PR on Current Repository by Default (#8670)
* Improve Open Graph Protocol (#8637)
* Batch hook pre- and post-receive calls (#8602)
* Improve webhooks (#8583)
* Move transfer repository and rename repository on a service package and start action notification (#8573)
* Implement/Fix PR review webhooks (#8570)
* Rewrite markdown rendering to blackfriday v2 and rewrite orgmode rendering to go-org (#8560)
* Move some repositories' operations to a standalone service package (#8557)
* Allow more than 255 characters for tokens in external_login_user table (#8554)
* Move issue label operations to issue service package (#8553)
* Adjust error reporting from merge failures and use LC_ALL=C for git (#8548)
* Mail assignee when issue/pull request is assigned (#8546)
* Allow committing / adding empty files using the web ui (#8420) (#8532)
* Move sync mirror actions to mirror service package (#8518)
* Remove arrows on numeric inputs (#8516)
* Support inline rendering of CUSTOM_URL_SCHEMES (#8496)
* Recalculate repository access only for specific user (#8481)
* Add download button for rull request diff- and patch-file (#8470)
* Add single sign-on support via SSPI on Windows (#8463)
* Move change issue title from models to issue service package (#8456)
* Add included tag on branch view (#8449)
* Make static resouces web browser cache time customized on app.ini (#8442)
* Enable Uploading/Removing Attachments When Editing an Issue/Comment (#8426)
* Add pagination to commit graph page (#8360)
* Use templates for issue e-mail subject and body (#8329)
* Move clearlabels from models to issue service (#8326)
* Move AddTestPullRequestTask to pull service package from models (#8324)
* Team permission to create repository in organization (#8312)
* Allows external rendering of other filetypes (#8300)
* Add 'Alt + click' feature to exclude labels (#8199)
* Configurable close and reopen keywords for PRs (#8120)
* Configurable URL for static resources (#7911)
* Unifies commit list in repository commit table and wiki revision page (#7907)
* Allow cross-repository dependencies on issues (#7901)
* Auto-subscribe user to repository when they commit/tag to it (#7657)
* Restore Graceful Restarting & Socket Activation (#7274)
* wiki - add 'write' 'preview' buttons to wiki edit like in issues (#7241)
* Change target branch for pull request (#6488)
* Display PR commits and diffs using base repo rather than forked (#3648)
* SECURITY
* Swagger hide search field (#9554)
* Add "search" to reserved usernames (#9063)
* Switch to fomantic-ui (#9374)
* Only serve attachments when linked to issue/release and if accessible by user (#9340)
* Hide credentials when submitting migration through API (#9102)
* TESTING
* Add debug option to serv to help debug problems (#9492)
* Fix the intermittent TestGPGGit failures (#9360)
* Testing: Update postgres sequences (#9304)
* Missed defer prepareTestEnv (#9285)
* Fix "data race" in testlogger (#9159)
* Yet another attempt to fix the intermittent failure of gpg git test (#9146)
* integrations: Fix Dropped Test Errors (#9040)
* services/mirror: fix dropped test errors (#9007)
* Fix intermittent GPG Git test failure (#8968)
* Update Github Migration Tests (#8893) (#8938)
* Update heatmap fixtures to restore tests (#8615)
* TRANSLATION
* Fix placeholders in the error message (#9060)
* Fix spelling of admin.users.max_repo_creation (#8934)
* Improve german translation of homepage (#8549)
* BUILD
* Update gitea.com/macaron to 1.4.0 (#9608)
* Upgrade lato fonts to v16. (#9498)
* Update alpine to 3.11 (#9440)
* Upgrade blevesearch (#9177)
* Remove built js/css files from git (#9114)
* Move semantic.dropdown.custom.js to webpack (#9064)
* Check compiled files during build (#9042)
* Enable lazy-loading of gitgraph.js (#9036)
* Pack web_src/js/draw.js to public/js/index.js (#8975)
* Modernize js and use babel (#8973)
* Move index.js to web_src and use webpack to pack them (#8598)
* Restrict modules/graceful to non-windows build and shim IsChild (#8537)
* Upgrade gopkg.in/editorconfig/editorconfig-core-go.v1 (#8501)
* DOCS
* Swagger info corrections (#9441) (#9558)
* Add ALLOW_ONLY_EXTERNAL_REGISTRATION to config cheat sheet (#8986)
* Rephrase comment about RuntimeDirectory option in systemd config (#8912)
* Explicitly indicate the socket unit to use the service unit "gitea.service" (#8804)
* Adjust the must-change-password help (#8755)
* Add notice to docs for migrating from more recent versions of Gogs (#8724)
* Add explicit info about customization of homepage (#8694)
* Change external asciidoctor tool to embedded mode (#8677)
* Add Docker fail2ban configuration (#8642)
* Correct some outdated statements in the contributing guidelines (#8612)
* Basic Design guidelines (describing different parts of the code) (#8601)
* Display Gitea logo in Readme (#8592)
* Fix building from source docs to ref AppWorkPath (#8567)
* Update the provided gitea.service to mention socket activation (#8531)
* Doc added how to setup email (#8520)
* MISC
* Add translatable Powered by Gitea text in footer (#9600)
* Add contrib/environment-to-ini (#9519)
* Remove unnecessary loading of settings in update hook (#9496)
* Update gitignore list (#9437)
* Update license list (#9436)
* Fix background reactions in the arc-green theme (#9421)
* Update and fix chardet import (#9351)
* Ensure LF on checkouts and in editors (#9259)
* Fixed topics margin (#9248)
* Add comment to exported function WindowsServiceName (make revive) (#9241)
* Remove empty lines on issues/pulls page (#9232)
* Fix Add Comment Button's "+" Position (#9140)
* Add first issue comment hashtag (#9052)
* Change some label colors (#9051)
* Fix double scroll in branch dropdown (#9048)
* Add comment highlight when target from url (#9047)
* Update display of reactions to issues and comments (#9038)
* Button tooltip formatting under Branches (#9034)
* Allow setting default branch via API (#9030)
* Update dashboard context for PR reviews (#8995)
* Show repository size in repo home page and settings (#8940)
* Allow to add and remove all repositories to/from team. (#8867)
* Show due date in dashboard issues list (#8860)
* Theme arc-green: reverse heatmap colors (#8840)
* Project files table style update (#8757)
* gitignore debugging file from vscode (#8740)
* Add API for Issue set Subscription (#8729)
* Make 100% width search bar (#8710)
* Update color theme for heatmap (#8709)
* Add margin to title_wip_desc (#8705)
* Improve visibility of "Pending" indicator (#8685)
* Improve accessibility of dropdown menus (#8638)
* Make /users/{username}/repos list private repos the current user has access to (#8621)
* Prevent .code-view from overriding font on icon fonts (#8614)
* Add id references on all issue events to allow internal linking (#8608)
* Upgrade xorm to v0.8.0 (#8536)
* Upgrade gopkg.in/ini.v1 (#8500)
* Update CodeMirror to version 5.49.0 (#8381)
* Wiki editor: enable side-by-side button (#7242)
## [1.10.2](https://github.com/go-gitea/gitea/releases/tag/v1.10.2) - 2020-01-02
* BUGFIXES
* Allow only specific Columns to be updated on Issue via API (#9539) (#9580)
@ -59,7 +439,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Be more strict with git arguments (#7715)
* Extract the username and password from the mirror url (#7651)
* reserve .well-known username (#7637)
* FEATURE
* FEATURES
* Org/Members: display 2FA members states + optimize sql requests (#7621)
* SetDefaultBranch on pushing to empty repository (#7610)
* Adds side-by-side diff for images (#6784)
@ -68,7 +448,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Add option to initialize repository with labels (#6061)
* Add additional password hash algorithms (#6023)
* BUGFIXES
* Allow to merge if file path contains " or \ (#8629) (#8771)
* Allow to merge if file path contains " or \ (#8629) (#8771)
* On windows set core.longpaths true (#8776) (#8786)
* Fix 500 when edit hook (#8782) (#8789)
* Fix Checkbox at RepoSettings Protected Branch (#8799) (#8801)
@ -80,7 +460,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Fix require external registration password (#8885) (#8890)
* Fix password complexity check on registration (#8887) (#8888)
* Update Github Migration Tests (#8896) (#8938) (#8945)
* Enable punctuations ending mentions (#8889) (#8894)
* Enable punctuations ending mentions (#8889) (#8894)
* Add Close() method to gogitRepository (#8901) (#8956)
* Hotfix for review actions and notifications (#8965)
* Expose db.SetMaxOpenConns and allow non MySQL dbs to set conn pool params (#8528) (#8618)
@ -218,7 +598,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* fix post parameter - on issue list - unset assignee (#7380)
* fix/define autochecked checkboxes on issue list in firefox (#7320)
* only return head: null if source branch was deleted (#6705)
* ENHANCEMENT
* ENHANCEMENTS
* Add nofollow to sign in links (#8509)
* vendor: update mvdan.cc/xurls/v2 to v2.1.0 (#8495)
* Update milestone issues numbers when save milestone and other code improvements (#8411)
@ -412,7 +792,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Fix adding default Telegram webhook (#7972) (#7992)
* Abort synchronization from LDAP source if there is some error (#7965)
* Fix deformed emoji in commit message (#8071)
* ENHANCEMENT
* ENHANCEMENTS
* Keep blame view buttons sequence consistent with normal view when viewing a file (#8007) (#8009)
## [1.9.2](https://github.com/go-gitea/gitea/releases/tag/v1.9.2) - 2019-08-22
@ -424,7 +804,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* SECURITY
* Fix No PGP signature on 1.9.1 tag (#7874)
* Release built with go 1.12.9 to fix security fixes in golang std lib, ref: https://groups.google.com/forum/#!msg/golang-announce/oeMaeUnkvVE/a49yvTLqAAAJ
* ENHANCEMENT
* ENHANCEMENTS
* Fix pull creation with empty changes (#7920) (#7926)
* BUILD
* Drone/docker: prepare multi-arch release + provide arm64 image (#7571) (#7884)
@ -465,7 +845,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Reserve .well-known username (#7638)
* Do not leak secrets via timing side channel (#7364)
* Ensure that decryption of cookie actually suceeds (#7363)
* FEATURE
* FEATURES
* Content API for Creating, Updating, Deleting Files (#6314)
* Enable tls-alpn-01: Use certmanager provided TLSConfig for LetsEncrypt (#7229)
* Add command to convert mysql database from utf8 to utf8mb4 (#7144)
@ -656,7 +1036,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Fix bug manifest.json will not request with cookie so that session will created every request (#6372)
* Disable benchmarking during tag events on DroneIO (#6365)
* Comments list performance optimization (#5305)
* ENHANCEMENT
* ENHANCEMENTS
* Update Drone docker generation to standard format (#7480) (#7496) (#7504)
* Add API Endpoint for Repo Edit (#7006)
* Add state param to milestone listing API (#7131)
@ -876,7 +1256,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Releases API paging (#5831)
* Allow Macaron to be set to log through to gitea.log (#5667)
* Don't close issues via commits on non-default branch (#5622)
* FEATURE
* FEATURES
* Add regenerate secret feature for oauth2 (#6291)
* Expose issue stopwatch toggling via API (#5970)
* Add other session providers (#5963)
@ -887,7 +1267,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Discord Oauth2 support (#4476)
* Allow to set organization visibility (public, internal, private) (#1763)
* Added URL mapping for Release attachments like on github.com (#1707)
* ENHANCEMENT
* ENHANCEMENTS
* Add support for client basic auth for exchanging access tokens (#6293)
* Add ability to sort issues by due date (#6206) (#6244)
* Style tweaks to issue selection (#6196)
@ -1177,7 +1557,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* BREAKING
* Restrict permission check on repositories and fix some problems (#5314)
* Show only opened milestones on issues page milestone filter (#5051)
* FEATURE
* FEATURES
* Implement git refs API for listing references (branches, tags and other) (#5354)
* Approvals at Branch Protection (#5350)
* Add raw blob endpoint to get objects by SHA ID (#5334)
@ -1262,7 +1642,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* LDAP via simple auth separate bind user and search base (#5055)
* Fix markdown image with link (#4675)
* Fix to 3819 - Filtering issues by tags on main screen issues (#3824)
* ENHANCEMENT
* ENHANCEMENTS
* Delete organization endpoint added (#5601)
* Update Licenses (#5558)
* Support reverse proxy providing email (#5554)
@ -1372,7 +1752,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Make cookies HttpOnly and obey COOKIE_SECURE flag (#4706)
* Don't disclose emails of all users when sending out emails (#4664)
* Check that repositories can only be migrated to own user or organizations (#4366)
* FEATURE
* FEATURES
* Add comment replies (#5147) (#5104)
* Pull request review/approval and comment on code (#3748)
* Added dependencies for issues (#2196) (#2531)
@ -1385,7 +1765,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Add push webhook support for mirrored repositories (#4127)
* Add csv file render support defaultly (#4105)
* Add Recaptcha functionality to Gitea (#4044)
* ENHANCEMENT
* ENHANCEMENTS
* Fix milestones sorted wrongly (#4987)
* Allow api to create tags for releases if they don't exist (#4890)
* Fix #4877 to follow the OpenID Connect Audiences spec (#4878)
@ -1567,7 +1947,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Fix bugs when too many IN variables (#4594) (#4597)
* Push whitelist now doesn't apply to branch deletion (#4601) (#4640)
* Site admin could create repos even MAX_CREATION_LIMIT=0 (#4645) (#4650)
* FEATURE
* FEATURES
* Add cli commands to regen hooks & keys (#3979)
* Add support for FIDO U2F (#3971)
* Added user language setting (#3875)
@ -1581,7 +1961,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Add repository setting to enable/disable health checks (#3607)
* Emoji Autocomplete (#3433)
* Implements generator cli for secrets (#3531)
* ENHANCEMENT
* ENHANCEMENTS
* Add more webhooks support and refactor webhook templates directory (#3929)
* Add new option to allow only OAuth2/OpenID user registration (#3910)
* Add option to use paged LDAP search when synchronizing users (#3895)
@ -1665,7 +2045,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Fix escaping changed title in comments (#3530) (#3534)
 * Escape search query (#3486) (#3488)
* Sanitize logs for mirror sync (#3057)
* FEATURE
* FEATURES
* Serve .patch and .diff for pull requests (#3305, #3293)
* Add repo-sync-releases admin command (#3254)
* Support default private when creating or migrating repository (#3239)
@ -1728,7 +2108,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Fix go-get, src and raw urls to new scheme (#2978)
* Fix error when add user has full name to team (#2973)
* Fix memcache support when value is returned as string always (#2924)
* ENHANCEMENT
* ENHANCEMENTS
* Use GiteaServer as the user agent for http requests (#3404)
* Delete indexer DB entries when (re)creating index (#3385)
* Change how merged PR commit info are prepared (#3368)
@ -1787,7 +2167,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
## [1.3.0](https://github.com/go-gitea/gitea/releases/tag/v1.3.0) - 2017-11-29
* BREAKING
* Make URL scheme unambiguous (#2408)
* FEATURE
* FEATURES
* Add branch overiew page (#2108)
* Code/repo search (#2582)
* Add Activity page to repository (#2674)
@ -1891,7 +2271,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Fix typos in app.ini (#2732)
* Fix duplicated rel attribute (#2549)
* Fix tests code to prevent some runtime errors (#2381)
* ENHANCEMENT
* ENHANCEMENTS
* Memory usage improvements and lower minimal git requirement to 1.7.2 (#3013) (#3028)
* Set OpenID support on by default when installing new instance (#3010) (#3027)
* Use api.TrackedTime in API (#2807)
@ -2037,7 +2417,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Sanitation fix from Gogs (#1461)
* BREAKING
* Rename /forget_password url to /forgot_password (#1219)
* FEATURE
* FEATURES
* Logo: Add task to generate images from SVG and change to new logo (#2194)
* Status-API (#1332)
* Show commit status icon in commits table (#1688)
@ -2064,7 +2444,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Add change-password admin command (#1304)
* Only use issue and wiki on repo. (#1297)
* Allow push to init a wiki repo (#1279)
* ENHANCEMENT
* ENHANCEMENTS
* Make time diff translatable (#2057)
* Smaller watch, star, and fork buttons (#2052)
* Display config file path on admin panel (#2030)
@ -2559,7 +2939,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* BREAKING
* The SSH keys can potentially break, make sure to regenerate the authorized keys
* FEATURE
* FEATURES
* Git LFSv2 support [#122](https://github.com/go-gitea/gitea/pull/122)
* API endpoints for repo watching [#191](https://github.com/go-gitea/gitea/pull/191)
* Search within private repos [#222](https://github.com/go-gitea/gitea/pull/222)
@ -2611,7 +2991,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Don't rewrite non-gitea public keys [#906](https://github.com/go-gitea/gitea/pull/906)
* Use fingerprint to check instead content for public key [#911](https://github.com/go-gitea/gitea/pull/911)
* Fix random avatars [#1147](https://github.com/go-gitea/gitea/pull/1147)
* ENHANCEMENT
* ENHANCEMENTS
* Refactored process manager [#75](https://github.com/go-gitea/gitea/pull/75)
* Restrict rights to create new orgs [#193](https://github.com/go-gitea/gitea/pull/193)
* Added label and milestone sorting [#199](https://github.com/go-gitea/gitea/pull/199)
@ -2673,7 +3053,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* BREAKING
* We have various changes on the API, scripting against API must be updated
* FEATURE
* FEATURES
* Show last login for admins [#121](https://github.com/go-gitea/gitea/pull/121)
* BUGFIXES
* Fixed sender of notifications [#2](https://github.com/go-gitea/gitea/pull/2)
@ -2686,7 +3066,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Replace tabs with spaces on wiki title [#371](https://github.com/go-gitea/gitea/pull/371)
* Fixed vulnerability on labels and releases [#409](https://github.com/go-gitea/gitea/pull/409)
* Fixed issue comment API [#449](https://github.com/go-gitea/gitea/pull/449)
* ENHANCEMENT
* ENHANCEMENTS
* Use proper import path for libravatar [#3](https://github.com/go-gitea/gitea/pull/3)
* Integrated DroneCI for tests and builds [#24](https://github.com/go-gitea/gitea/issues/24)
* Integrated dependency manager [#29](https://github.com/go-gitea/gitea/issues/29)

2
CONTRIBUTING.md
View file

@ -354,7 +354,7 @@ be reviewed by two maintainers and must pass the automatic tests.
Code that you contribute should use the standard copyright header:
```
// Copyright 2019 The Gitea Authors. All rights reserved.
// Copyright 2020 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
```

5
cmd/admin.go
View file

@ -155,6 +155,7 @@ var (
microcmdAuthDelete = cli.Command{
Name: "delete",
Usage: "Delete specific auth source",
Flags: []cli.Flag{idFlag},
Action: runDeleteAuth,
}
@ -533,9 +534,9 @@ func runListAuth(c *cli.Context) error {
// loop through each source and print
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', tabwriter.AlignRight)
fmt.Fprintf(w, "ID\tName\tType\tEnabled")
fmt.Fprintf(w, "ID\tName\tType\tEnabled\n")
for _, source := range loginSources {
fmt.Fprintf(w, "%d\t%s\t%s\t%t", source.ID, source.Name, models.LoginNames[source.Type], source.IsActived)
fmt.Fprintf(w, "%d\t%s\t%s\t%t\n", source.ID, source.Name, models.LoginNames[source.Type], source.IsActived)
}
w.Flush()

33
custom/conf/app.ini.sample
View file

@ -382,6 +382,39 @@ REPO_INDEXER_INCLUDE =
; A comma separated list of glob patterns to exclude from the index; ; default is empty
REPO_INDEXER_EXCLUDE =
[queue]
; Specific queues can be individually configured with [queue.name]. [queue] provides defaults
;
; General queue queue type, currently support: persistable-channel, channel, level, redis, dummy
; default to persistable-channel
TYPE = persistable-channel
; data-dir for storing persistable queues and level queues, individual queues will be named by their type
DATADIR = queues/
; Default queue length before a channel queue will block
LENGTH = 20
; Batch size to send for batched queues
BATCH_LENGTH = 20
; Connection string for redis queues this will store the redis connection string.
CONN_STR = "addrs=127.0.0.1:6379 db=0"
; Provide the suffix of the default redis queue name - specific queues can be overriden within in their [queue.name] sections.
QUEUE_NAME = "_queue"
; If the queue cannot be created at startup - level queues may need a timeout at startup - wrap the queue:
WRAP_IF_NECESSARY = true
; Attempt to create the wrapped queue at max
MAX_ATTEMPTS = 10
; Timeout queue creation
TIMEOUT = 15m30s
; Create a pool with this many workers
WORKERS = 1
; Dynamically scale the worker pool to at this many workers
MAX_WORKERS = 10
; Add boost workers when the queue blocks for BLOCK_TIMEOUT
BLOCK_TIMEOUT = 1s
; Remove the boost workers after BOOST_TIMEOUT
BOOST_TIMEOUT = 5m
; During a boost add BOOST_WORKERS
BOOST_WORKERS = 5
[admin]
; Disallow regular (non-admin) users from creating organizations.
DISABLE_REGULAR_ORG_CREATION = false

20
docs/content/doc/advanced/config-cheat-sheet.en-us.md
View file

@ -226,6 +226,7 @@ relation to port exhaustion.
- `ISSUE_INDEXER_TYPE`: **bleve**: Issue indexer type, currently support: bleve or db, if it's db, below issue indexer item will be invalid.
- `ISSUE_INDEXER_PATH`: **indexers/issues.bleve**: Index file used for issue search.
- The next 4 configuration values are deprecated and should be set in `queue.issue_indexer` however are kept for backwards compatibility:
- `ISSUE_INDEXER_QUEUE_TYPE`: **levelqueue**: Issue indexer queue, currently supports:`channel`, `levelqueue`, `redis`.
- `ISSUE_INDEXER_QUEUE_DIR`: **indexers/issues.queue**: When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this will be the queue will be saved path.
- `ISSUE_INDEXER_QUEUE_CONN_STR`: **addrs=127.0.0.1:6379 db=0**: When `ISSUE_INDEXER_QUEUE_TYPE` is `redis`, this will store the redis connection string.
@ -239,6 +240,24 @@ relation to port exhaustion.
- `MAX_FILE_SIZE`: **1048576**: Maximum size in bytes of files to be indexed.
- `STARTUP_TIMEOUT`: **30s**: If the indexer takes longer than this timeout to start - fail. (This timeout will be added to the hammer time above for child processes - as bleve will not start until the previous parent is shutdown.) Set to zero to never timeout.
## Queue (`queue` and `queue.*`)
- `TYPE`: **persistable-channel**: General queue type, currently support: `persistable-channel`, `channel`, `level`, `redis`, `dummy`
- `DATADIR`: **queues/**: Base DataDir for storing persistent and level queues. `DATADIR` for inidividual queues can be set in `queue.name` sections but will default to `DATADIR/`**`name`**.
- `LENGTH`: **20**: Maximal queue size before channel queues block
- `BATCH_LENGTH`: **20**: Batch data before passing to the handler
- `CONN_STR`: **addrs=127.0.0.1:6379 db=0**: Connection string for the redis queue type.
- `QUEUE_NAME`: **_queue**: The suffix for default redis queue name. Individual queues will default to **`name`**`QUEUE_NAME` but can be overriden in the specific `queue.name` section.
- `WRAP_IF_NECESSARY`: **true**: Will wrap queues with a timeoutable queue if the selected queue is not ready to be created - (Only relevant for the level queue.)
- `MAX_ATTEMPTS`: **10**: Maximum number of attempts to create the wrapped queue
- `TIMEOUT`: **GRACEFUL_HAMMER_TIME + 30s**: Timeout the creation of the wrapped queue if it takes longer than this to create.
- Queues by default come with a dynamically scaling worker pool. The following settings configure this:
- `WORKERS`: **1**: Number of initial workers for the queue.
- `MAX_WORKERS`: **10**: Maximum number of worker go-routines for the queue.
- `BLOCK_TIMEOUT`: **1s**: If the queue blocks for this time, boost the number of workers - the `BLOCK_TIMEOUT` will then be doubled before boosting again whilst the boost is ongoing.
- `BOOST_TIMEOUT`: **5m**: Boost workers will timeout after this long.
- `BOOST_WORKERS`: **5**: This many workers will be added to the worker pool if there is a boost.
## Admin (`admin`)
- `DEFAULT_EMAIL_NOTIFICATIONS`: **enabled**: Default configuration for email notifications for users (user configurable). Options: enabled, onmention, disabled
@ -614,6 +633,7 @@ You may redefine `ELEMENT`, `ALLOW_ATTR`, and `REGEXP` multiple times; each time
## Task (`task`)
- Task queue configuration has been moved to `queue.task` however, the below configuration values are kept for backwards compatibilityx:
- `QUEUE_TYPE`: **channel**: Task queue type, could be `channel` or `redis`.
- `QUEUE_LENGTH`: **1000**: Task queue length, available only when `QUEUE_TYPE` is `channel`.
- `QUEUE_CONN_STR`: **addrs=127.0.0.1:6379 db=0**: Task queue connection string, available only when `QUEUE_TYPE` is `redis`. If there redis needs a password, use `addrs=127.0.0.1:6379 password=123 db=0`.

6
docs/content/doc/advanced/customizing-gitea.en-us.md
View file

@ -44,12 +44,6 @@ environment variable; this can be used to override the default path to something
**Note:** Gitea must perform a full restart to see configuration changes.
## Customizing /robots.txt
To make Gitea serve a custom `/robots.txt` (default: empty 404), create a file called
`robots.txt` in the `custom` folder (or `CustomPath`) with
[expected contents](http://www.robotstxt.org/).
## Serving custom public files
To make Gitea serve custom public files (like pages and images), use the folder

39
docs/content/doc/advanced/search-engines-indexation.en-us.md Normal file
View file

@ -0,0 +1,39 @@
---
date: "2019-12-31T13:55:00+05:00"
title: "Advanced: Search Engines Indexation"
slug: "search-engines-indexation"
weight: 30
toc: true
draft: false
menu:
sidebar:
parent: "advanced"
name: "Search Engines Indexation"
weight: 60
identifier: "search-engines-indexation"
---
# Search engines indexation of your Gitea installation
By default your Gitea installation will be indexed by search engines.
If you don't want your repository to be visible for search engines read further.
## Block search engines indexation using robots.txt
To make Gitea serve a custom `robots.txt` (default: empty 404) for top level installations,
create a file called `robots.txt` in the [`custom` folder or `CustomPath`]({{< relref "doc/advanced/customizing-gitea.en-us.md" >}})
Examples on how to configure the `robots.txt` can be found at [https://moz.com/learn/seo/robotstxt](https://moz.com/learn/seo/robotstxt).
```txt
User-agent: *
Disallow: /
```
If you installed Gitea in a subdirectory, you will need to create or edit the `robots.txt` in the top level directory.
```txt
User-agent: *
Disallow: /gitea/
```

2
docs/content/doc/usage/https-support.md
View file

@ -76,4 +76,4 @@ After that, enable HTTPS by following one of these guides:
* [apache2/httpd](https://httpd.apache.org/docs/2.4/ssl/ssl_howto.html)
* [caddy](https://caddyserver.com/docs/tls)
Note: Enabling HTTPS only at the proxy level is referred as [TLS Termination Proxy](https://en.wikipedia.org/wiki/TLS_termination_proxy). The proxy server accepted incoming TLS connections, decrypts the contents, and pass the now unencrypted contents to Gitea. This is normally fine as long as both the proxy and Gitea instances are either on the same machine, or on different machines within private network (with the proxy is exposed to outside network. If your Gitea instance is separated from your proxy over a public network, or if you want full end-to-end encryption, you can also [enable HTTPS support directly in Gitea using built-in server](#using-the-built-in-server) and forward the connections over HTTPS instead.
Note: Enabling HTTPS only at the proxy level is referred as [TLS Termination Proxy](https://en.wikipedia.org/wiki/TLS_termination_proxy). The proxy server accepts incoming TLS connections, decrypts the contents, and passes the now unencrypted contents to Gitea. This is normally fine as long as both the proxy and Gitea instances are either on the same machine, or on different machines within private network (with the proxy is exposed to outside network). If your Gitea instance is separated from your proxy over a public network, or if you want full end-to-end encryption, you can also [enable HTTPS support directly in Gitea using built-in server](#using-the-built-in-server) and forward the connections over HTTPS instead.

27
integrations/api_comment_test.go
View file

@ -83,6 +83,33 @@ func TestAPICreateComment(t *testing.T) {
models.AssertExistsAndLoadBean(t, &models.Comment{ID: updatedComment.ID, IssueID: issue.ID, Content: commentBody})
}
func TestAPIGetComment(t *testing.T) {
defer prepareTestEnv(t)()
comment := models.AssertExistsAndLoadBean(t, &models.Comment{ID: 2}).(*models.Comment)
assert.NoError(t, comment.LoadIssue())
repo := models.AssertExistsAndLoadBean(t, &models.Repository{ID: comment.Issue.RepoID}).(*models.Repository)
repoOwner := models.AssertExistsAndLoadBean(t, &models.User{ID: repo.OwnerID}).(*models.User)
session := loginUser(t, repoOwner.Name)
token := getTokenForLoggedInUser(t, session)
req := NewRequestf(t, "GET", "/api/v1/repos/%s/%s/issues/comments/%d", repoOwner.Name, repo.Name, comment.ID)
resp := session.MakeRequest(t, req, http.StatusOK)
req = NewRequestf(t, "GET", "/api/v1/repos/%s/%s/issues/comments/%d?token=%s", repoOwner.Name, repo.Name, comment.ID, token)
resp = session.MakeRequest(t, req, http.StatusOK)
var apiComment api.Comment
DecodeJSON(t, resp, &apiComment)
assert.NoError(t, comment.LoadPoster())
expect := comment.APIFormat()
assert.Equal(t, expect.ID, apiComment.ID)
assert.Equal(t, expect.Poster.FullName, apiComment.Poster.FullName)
assert.Equal(t, expect.Body, apiComment.Body)
assert.Equal(t, expect.Created.Unix(), apiComment.Created.Unix())
}
func TestAPIEditComment(t *testing.T) {
defer prepareTestEnv(t)()
const newCommentBody = "This is the new comment body"

BIN
integrations/gitea-repositories-meta/user2/repo1.git/objects/00/750edc07d6415dcc07ae0351e9397b0222b7ba Normal file
View file

Binary file not shown.

BIN
integrations/gitea-repositories-meta/user2/repo1.git/objects/4a/357436d925b5c974181ff12a994538ddc5a269 Normal file
View file

Binary file not shown.

BIN
integrations/gitea-repositories-meta/user2/repo1.git/objects/dc/7a8ba127fee870dd683310ce660dfe59333a1b Normal file
View file

Binary file not shown.

1
integrations/gitea-repositories-meta/user2/repo1.git/refs/pull/3/head Normal file
View file

@ -0,0 +1 @@
4a357436d925b5c974181ff12a994538ddc5a269

9
integrations/issue_test.go
View file

@ -11,8 +11,10 @@ import (
"strconv"
"strings"
"testing"
"time"
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/indexer/issues"
"code.gitea.io/gitea/modules/references"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/test"
@ -87,7 +89,12 @@ func TestViewIssuesKeyword(t *testing.T) {
defer prepareTestEnv(t)()
repo := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 1}).(*models.Repository)
issue := models.AssertExistsAndLoadBean(t, &models.Issue{
RepoID: repo.ID,
Index: 1,
}).(*models.Issue)
issues.UpdateIssueIndexer(issue)
time.Sleep(time.Second * 1)
const keyword = "first"
req := NewRequestf(t, "GET", "%s/issues?q=%s", repo.RelLink(), keyword)
resp := MakeRequest(t, req, http.StatusOK)

9
models/issue_watch.go
View file

@ -4,7 +4,9 @@
package models
import "code.gitea.io/gitea/modules/timeutil"
import (
"code.gitea.io/gitea/modules/timeutil"
)
// IssueWatch is connection request for receiving issue notification.
type IssueWatch struct {
@ -46,17 +48,18 @@ func CreateOrUpdateIssueWatch(userID, issueID int64, isWatching bool) error {
return nil
}
// GetIssueWatch returns an issue watch by user and issue
// GetIssueWatch returns all IssueWatch objects from db by user and issue
// the current Web-UI need iw object for watchers AND explicit non-watchers
func GetIssueWatch(userID, issueID int64) (iw *IssueWatch, exists bool, err error) {
return getIssueWatch(x, userID, issueID)
}
// Return watcher AND explicit non-watcher if entry in db exist
func getIssueWatch(e Engine, userID, issueID int64) (iw *IssueWatch, exists bool, err error) {
iw = new(IssueWatch)
exists, err = e.
Where("user_id = ?", userID).
And("issue_id = ?", issueID).
And("is_watching = ?", true).
Get(iw)
return
}

5
models/issue_watch_test.go
View file

@ -29,9 +29,10 @@ func TestGetIssueWatch(t *testing.T) {
assert.True(t, exists)
assert.NoError(t, err)
_, exists, err = GetIssueWatch(2, 2)
assert.False(t, exists)
iw, exists, err := GetIssueWatch(2, 2)
assert.True(t, exists)
assert.NoError(t, err)
assert.EqualValues(t, false, iw.IsWatching)
_, exists, err = GetIssueWatch(3, 1)
assert.False(t, exists)

2
models/pull.go
View file

@ -122,7 +122,7 @@ func (pr *PullRequest) LoadHeadRepo() error {
if has, err := x.ID(pr.HeadRepoID).Get(&repo); err != nil {
return err
} else if !has {
return ErrRepoNotExist{ID: pr.BaseRepoID}
return ErrRepoNotExist{ID: pr.HeadRepoID}
}
pr.HeadRepo = &repo
}

313
models/wiki.go
View file

@ -5,53 +5,12 @@
package models
import (
"fmt"
"net/url"
"os"
"path/filepath"
"strings"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/sync"
"github.com/unknwon/com"
)
var (
reservedWikiNames = []string{"_pages", "_new", "_edit", "raw"}
wikiWorkingPool = sync.NewExclusivePool()
)
// NormalizeWikiName normalizes a wiki name
func NormalizeWikiName(name string) string {
return strings.Replace(name, "-", " ", -1)
}
// WikiNameToSubURL converts a wiki name to its corresponding sub-URL.
func WikiNameToSubURL(name string) string {
return url.QueryEscape(strings.Replace(name, " ", "-", -1))
}
// WikiNameToFilename converts a wiki name to its corresponding filename.
func WikiNameToFilename(name string) string {
name = strings.Replace(name, " ", "-", -1)
return url.QueryEscape(name) + ".md"
}
// WikiFilenameToName converts a wiki filename to its corresponding page name.
func WikiFilenameToName(filename string) (string, error) {
if !strings.HasSuffix(filename, ".md") {
return "", ErrWikiInvalidFileName{filename}
}
basename := filename[:len(filename)-3]
unescaped, err := url.QueryUnescape(basename)
if err != nil {
return "", err
}
return NormalizeWikiName(unescaped), nil
}
// WikiCloneLink returns clone URLs of repository wiki.
func (repo *Repository) WikiCloneLink() *CloneLink {
return repo.cloneLink(x, true)
@ -71,275 +30,3 @@ func (repo *Repository) WikiPath() string {
func (repo *Repository) HasWiki() bool {
return com.IsDir(repo.WikiPath())
}
// InitWiki initializes a wiki for repository,
// it does nothing when repository already has wiki.
func (repo *Repository) InitWiki() error {
if repo.HasWiki() {
return nil
}
if err := git.InitRepository(repo.WikiPath(), true); err != nil {
return fmt.Errorf("InitRepository: %v", err)
} else if err = createDelegateHooks(repo.WikiPath()); err != nil {
return fmt.Errorf("createDelegateHooks: %v", err)
}
return nil
}
// nameAllowed checks if a wiki name is allowed
func nameAllowed(name string) error {
for _, reservedName := range reservedWikiNames {
if name == reservedName {
return ErrWikiReservedName{name}
}
}
return nil
}
// updateWikiPage adds a new page to the repository wiki.
func (repo *Repository) updateWikiPage(doer *User, oldWikiName, newWikiName, content, message string, isNew bool) (err error) {
if err = nameAllowed(newWikiName); err != nil {
return err
}
wikiWorkingPool.CheckIn(com.ToStr(repo.ID))
defer wikiWorkingPool.CheckOut(com.ToStr(repo.ID))
if err = repo.InitWiki(); err != nil {
return fmt.Errorf("InitWiki: %v", err)
}
hasMasterBranch := git.IsBranchExist(repo.WikiPath(), "master")
basePath, err := CreateTemporaryPath("update-wiki")
if err != nil {
return err
}
defer func() {
if err := RemoveTemporaryPath(basePath); err != nil {
log.Error("Merge: RemoveTemporaryPath: %s", err)
}
}()
cloneOpts := git.CloneRepoOptions{
Bare: true,
Shared: true,
}
if hasMasterBranch {
cloneOpts.Branch = "master"
}
if err := git.Clone(repo.WikiPath(), basePath, cloneOpts); err != nil {
log.Error("Failed to clone repository: %s (%v)", repo.FullName(), err)
return fmt.Errorf("Failed to clone repository: %s (%v)", repo.FullName(), err)
}
gitRepo, err := git.OpenRepository(basePath)
if err != nil {
log.Error("Unable to open temporary repository: %s (%v)", basePath, err)
return fmt.Errorf("Failed to open new temporary repository in: %s %v", basePath, err)
}
defer gitRepo.Close()
if hasMasterBranch {
if err := gitRepo.ReadTreeToIndex("HEAD"); err != nil {
log.Error("Unable to read HEAD tree to index in: %s %v", basePath, err)
return fmt.Errorf("Unable to read HEAD tree to index in: %s %v", basePath, err)
}
}
newWikiPath := WikiNameToFilename(newWikiName)
if isNew {
filesInIndex, err := gitRepo.LsFiles(newWikiPath)
if err != nil {
log.Error("%v", err)
return err
}
for _, file := range filesInIndex {
if file == newWikiPath {
return ErrWikiAlreadyExist{newWikiPath}
}
}
} else {
oldWikiPath := WikiNameToFilename(oldWikiName)
filesInIndex, err := gitRepo.LsFiles(oldWikiPath)
if err != nil {
log.Error("%v", err)
return err
}
found := false
for _, file := range filesInIndex {
if file == oldWikiPath {
found = true
break
}
}
if found {
err := gitRepo.RemoveFilesFromIndex(oldWikiPath)
if err != nil {
log.Error("%v", err)
return err
}
}
}
// FIXME: The wiki doesn't have lfs support at present - if this changes need to check attributes here
objectHash, err := gitRepo.HashObject(strings.NewReader(content))
if err != nil {
log.Error("%v", err)
return err
}
if err := gitRepo.AddObjectToIndex("100644", objectHash, newWikiPath); err != nil {
log.Error("%v", err)
return err
}
tree, err := gitRepo.WriteTree()
if err != nil {
log.Error("%v", err)
return err
}
commitTreeOpts := git.CommitTreeOpts{
Message: message,
}
sign, signingKey := repo.SignWikiCommit(doer)
if sign {
commitTreeOpts.KeyID = signingKey
} else {
commitTreeOpts.NoGPGSign = true
}
if hasMasterBranch {
commitTreeOpts.Parents = []string{"HEAD"}
}
commitHash, err := gitRepo.CommitTree(doer.NewGitSig(), tree, commitTreeOpts)
if err != nil {
log.Error("%v", err)
return err
}
if err := git.Push(basePath, git.PushOptions{
Remote: "origin",
Branch: fmt.Sprintf("%s:%s%s", commitHash.String(), git.BranchPrefix, "master"),
Env: FullPushingEnvironment(
doer,
doer,
repo,
repo.Name+".wiki",
0,
),
}); err != nil {
log.Error("%v", err)
return fmt.Errorf("Push: %v", err)
}
return nil
}
// AddWikiPage adds a new wiki page with a given wikiPath.
func (repo *Repository) AddWikiPage(doer *User, wikiName, content, message string) error {
return repo.updateWikiPage(doer, "", wikiName, content, message, true)
}
// EditWikiPage updates a wiki page identified by its wikiPath,
// optionally also changing wikiPath.
func (repo *Repository) EditWikiPage(doer *User, oldWikiName, newWikiName, content, message string) error {
return repo.updateWikiPage(doer, oldWikiName, newWikiName, content, message, false)
}
// DeleteWikiPage deletes a wiki page identified by its path.
func (repo *Repository) DeleteWikiPage(doer *User, wikiName string) (err error) {
wikiWorkingPool.CheckIn(com.ToStr(repo.ID))
defer wikiWorkingPool.CheckOut(com.ToStr(repo.ID))
if err = repo.InitWiki(); err != nil {
return fmt.Errorf("InitWiki: %v", err)
}
basePath, err := CreateTemporaryPath("update-wiki")
if err != nil {
return err
}
defer func() {
if err := RemoveTemporaryPath(basePath); err != nil {
log.Error("Merge: RemoveTemporaryPath: %s", err)
}
}()
if err := git.Clone(repo.WikiPath(), basePath, git.CloneRepoOptions{
Bare: true,
Shared: true,
Branch: "master",
}); err != nil {
log.Error("Failed to clone repository: %s (%v)", repo.FullName(), err)
return fmt.Errorf("Failed to clone repository: %s (%v)", repo.FullName(), err)
}
gitRepo, err := git.OpenRepository(basePath)
if err != nil {
log.Error("Unable to open temporary repository: %s (%v)", basePath, err)
return fmt.Errorf("Failed to open new temporary repository in: %s %v", basePath, err)
}
defer gitRepo.Close()
if err := gitRepo.ReadTreeToIndex("HEAD"); err != nil {
log.Error("Unable to read HEAD tree to index in: %s %v", basePath, err)
return fmt.Errorf("Unable to read HEAD tree to index in: %s %v", basePath, err)
}
wikiPath := WikiNameToFilename(wikiName)
filesInIndex, err := gitRepo.LsFiles(wikiPath)
found := false
for _, file := range filesInIndex {
if file == wikiPath {
found = true
break
}
}
if found {
err := gitRepo.RemoveFilesFromIndex(wikiPath)
if err != nil {
return err
}
} else {
return os.ErrNotExist
}
// FIXME: The wiki doesn't have lfs support at present - if this changes need to check attributes here
tree, err := gitRepo.WriteTree()
if err != nil {
return err
}
message := "Delete page '" + wikiName + "'"
commitTreeOpts := git.CommitTreeOpts{
Message: message,
Parents: []string{"HEAD"},
}
sign, signingKey := repo.SignWikiCommit(doer)
if sign {
commitTreeOpts.KeyID = signingKey
} else {
commitTreeOpts.NoGPGSign = true
}
commitHash, err := gitRepo.CommitTree(doer.NewGitSig(), tree, commitTreeOpts)
if err != nil {
return err
}
if err := git.Push(basePath, git.PushOptions{
Remote: "origin",
Branch: fmt.Sprintf("%s:%s%s", commitHash.String(), git.BranchPrefix, "master"),
Env: PushingEnvironment(doer, repo),
}); err != nil {
return fmt.Errorf("Push: %v", err)
}
return nil
}

193
models/wiki_test.go
View file

@ -8,100 +8,11 @@ import (
"path/filepath"
"testing"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/setting"
"github.com/stretchr/testify/assert"
)
func TestNormalizeWikiName(t *testing.T) {
type test struct {
Expected string
WikiName string
}
for _, test := range []test{
{"wiki name", "wiki name"},
{"wiki name", "wiki-name"},
{"name with/slash", "name with/slash"},
{"name with%percent", "name-with%percent"},
{"%2F", "%2F"},
} {
assert.Equal(t, test.Expected, NormalizeWikiName(test.WikiName))
}
}
func TestWikiNameToFilename(t *testing.T) {
type test struct {
Expected string
WikiName string
}
for _, test := range []test{
{"wiki-name.md", "wiki name"},
{"wiki-name.md", "wiki-name"},
{"name-with%2Fslash.md", "name with/slash"},
{"name-with%25percent.md", "name with%percent"},
} {
assert.Equal(t, test.Expected, WikiNameToFilename(test.WikiName))
}
}
func TestWikiNameToSubURL(t *testing.T) {
type test struct {
Expected string
WikiName string
}
for _, test := range []test{
{"wiki-name", "wiki name"},
{"wiki-name", "wiki-name"},
{"name-with%2Fslash", "name with/slash"},
{"name-with%25percent", "name with%percent"},
} {
assert.Equal(t, test.Expected, WikiNameToSubURL(test.WikiName))
}
}
func TestWikiFilenameToName(t *testing.T) {
type test struct {
Expected string
Filename string
}
for _, test := range []test{
{"hello world", "hello-world.md"},
{"symbols/?*", "symbols%2F%3F%2A.md"},
} {
name, err := WikiFilenameToName(test.Filename)
assert.NoError(t, err)
assert.Equal(t, test.Expected, name)
}
for _, badFilename := range []string{
"nofileextension",
"wrongfileextension.txt",
} {
_, err := WikiFilenameToName(badFilename)
assert.Error(t, err)
assert.True(t, IsErrWikiInvalidFileName(err))
}
_, err := WikiFilenameToName("badescaping%%.md")
assert.Error(t, err)
assert.False(t, IsErrWikiInvalidFileName(err))
}
func TestWikiNameToFilenameToName(t *testing.T) {
// converting from wiki name to filename, then back to wiki name should
// return the original (normalized) name
for _, name := range []string{
"wiki-name",
"wiki name",
"wiki name with/slash",
"$$$%%%^^&&!@#$(),.<>",
} {
filename := WikiNameToFilename(name)
resultName, err := WikiFilenameToName(filename)
assert.NoError(t, err)
assert.Equal(t, NormalizeWikiName(name), resultName)
}
}
func TestRepository_WikiCloneLink(t *testing.T) {
assert.NoError(t, PrepareTestDatabase())
@ -131,107 +42,3 @@ func TestRepository_HasWiki(t *testing.T) {
repo2 := AssertExistsAndLoadBean(t, &Repository{ID: 2}).(*Repository)
assert.False(t, repo2.HasWiki())
}
func TestRepository_InitWiki(t *testing.T) {
PrepareTestEnv(t)
// repo1 already has a wiki
repo1 := AssertExistsAndLoadBean(t, &Repository{ID: 1}).(*Repository)
assert.NoError(t, repo1.InitWiki())
// repo2 does not already have a wiki
repo2 := AssertExistsAndLoadBean(t, &Repository{ID: 2}).(*Repository)
assert.NoError(t, repo2.InitWiki())
assert.True(t, repo2.HasWiki())
}
func TestRepository_AddWikiPage(t *testing.T) {
assert.NoError(t, PrepareTestDatabase())
const wikiContent = "This is the wiki content"
const commitMsg = "Commit message"
repo := AssertExistsAndLoadBean(t, &Repository{ID: 1}).(*Repository)
doer := AssertExistsAndLoadBean(t, &User{ID: 2}).(*User)
for _, wikiName := range []string{
"Another page",
"Here's a <tag> and a/slash",
} {
wikiName := wikiName
t.Run("test wiki exist: "+wikiName, func(t *testing.T) {
t.Parallel()
assert.NoError(t, repo.AddWikiPage(doer, wikiName, wikiContent, commitMsg))
// Now need to show that the page has been added:
gitRepo, err := git.OpenRepository(repo.WikiPath())
assert.NoError(t, err)
defer gitRepo.Close()
masterTree, err := gitRepo.GetTree("master")
assert.NoError(t, err)
wikiPath := WikiNameToFilename(wikiName)
entry, err := masterTree.GetTreeEntryByPath(wikiPath)
assert.NoError(t, err)
assert.Equal(t, wikiPath, entry.Name(), "%s not addded correctly", wikiName)
})
}
t.Run("check wiki already exist", func(t *testing.T) {
t.Parallel()
// test for already-existing wiki name
err := repo.AddWikiPage(doer, "Home", wikiContent, commitMsg)
assert.Error(t, err)
assert.True(t, IsErrWikiAlreadyExist(err))
})
t.Run("check wiki reserved name", func(t *testing.T) {
t.Parallel()
// test for reserved wiki name
err := repo.AddWikiPage(doer, "_edit", wikiContent, commitMsg)
assert.Error(t, err)
assert.True(t, IsErrWikiReservedName(err))
})
}
func TestRepository_EditWikiPage(t *testing.T) {
const newWikiContent = "This is the new content"
const commitMsg = "Commit message"
repo := AssertExistsAndLoadBean(t, &Repository{ID: 1}).(*Repository)
doer := AssertExistsAndLoadBean(t, &User{ID: 2}).(*User)
for _, newWikiName := range []string{
"Home", // same name as before
"New home",
"New/name/with/slashes",
} {
PrepareTestEnv(t)
assert.NoError(t, repo.EditWikiPage(doer, "Home", newWikiName, newWikiContent, commitMsg))
// Now need to show that the page has been added:
gitRepo, err := git.OpenRepository(repo.WikiPath())
assert.NoError(t, err)
masterTree, err := gitRepo.GetTree("master")
assert.NoError(t, err)
wikiPath := WikiNameToFilename(newWikiName)
entry, err := masterTree.GetTreeEntryByPath(wikiPath)
assert.NoError(t, err)
assert.Equal(t, wikiPath, entry.Name(), "%s not editted correctly", newWikiName)
if newWikiName != "Home" {
_, err := masterTree.GetTreeEntryByPath("Home.md")
assert.Error(t, err)
}
gitRepo.Close()
}
}
func TestRepository_DeleteWikiPage(t *testing.T) {
PrepareTestEnv(t)
repo := AssertExistsAndLoadBean(t, &Repository{ID: 1}).(*Repository)
doer := AssertExistsAndLoadBean(t, &User{ID: 2}).(*User)
assert.NoError(t, repo.DeleteWikiPage(doer, "Home"))
// Now need to show that the page has been added:
gitRepo, err := git.OpenRepository(repo.WikiPath())
assert.NoError(t, err)
defer gitRepo.Close()
masterTree, err := gitRepo.GetTree("master")
assert.NoError(t, err)
wikiPath := WikiNameToFilename("Home")
_, err = masterTree.GetTreeEntryByPath(wikiPath)
assert.Error(t, err)
}

4
modules/indexer/issues/db.go
View file

@ -25,6 +25,10 @@ func (db *DBIndexer) Delete(ids ...int64) error {
return nil
}
// Close dummy function
func (db *DBIndexer) Close() {
}
// Search dummy function
func (db *DBIndexer) Search(kw string, repoIDs []int64, limit, start int) (*SearchResult, error) {
total, ids, err := models.SearchIssueIDsByKeyword(kw, repoIDs, limit, start)

199
modules/indexer/issues/indexer.go
View file

@ -5,12 +5,16 @@
package issues
import (
"context"
"fmt"
"os"
"sync"
"time"
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/graceful"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/queue"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"
)
@ -44,12 +48,14 @@ type Indexer interface {
Index(issue []*IndexerData) error
Delete(ids ...int64) error
Search(kw string, repoIDs []int64, limit, start int) (*SearchResult, error)
Close()
}
type indexerHolder struct {
indexer Indexer
mutex sync.RWMutex
cond *sync.Cond
indexer Indexer
mutex sync.RWMutex
cond *sync.Cond
cancelled bool
}
func newIndexerHolder() *indexerHolder {
@ -58,6 +64,13 @@ func newIndexerHolder() *indexerHolder {
return h
}
func (h *indexerHolder) cancel() {
h.mutex.Lock()
defer h.mutex.Unlock()
h.cancelled = true
h.cond.Broadcast()
}
func (h *indexerHolder) set(indexer Indexer) {
h.mutex.Lock()
defer h.mutex.Unlock()
@ -68,16 +81,15 @@ func (h *indexerHolder) set(indexer Indexer) {
func (h *indexerHolder) get() Indexer {
h.mutex.RLock()
defer h.mutex.RUnlock()
if h.indexer == nil {
if h.indexer == nil && !h.cancelled {
h.cond.Wait()
}
return h.indexer
}
var (
issueIndexerChannel = make(chan *IndexerData, setting.Indexer.UpdateQueueLength)
// issueIndexerQueue queue of issue ids to be updated
issueIndexerQueue Queue
issueIndexerQueue queue.Queue
holder = newIndexerHolder()
)
@ -85,90 +97,99 @@ var (
// all issue index done.
func InitIssueIndexer(syncReindex bool) {
waitChannel := make(chan time.Duration)
// Create the Queue
switch setting.Indexer.IssueType {
case "bleve":
handler := func(data ...queue.Data) {
indexer := holder.get()
if indexer == nil {
log.Error("Issue indexer handler: unable to get indexer!")
return
}
iData := make([]*IndexerData, 0, setting.Indexer.IssueQueueBatchNumber)
for _, datum := range data {
indexerData, ok := datum.(*IndexerData)
if !ok {
log.Error("Unable to process provided datum: %v - not possible to cast to IndexerData", datum)
continue
}
log.Trace("IndexerData Process: %d %v %t", indexerData.ID, indexerData.IDs, indexerData.IsDelete)
if indexerData.IsDelete {
_ = indexer.Delete(indexerData.IDs...)
continue
}
iData = append(iData, indexerData)
}
if err := indexer.Index(iData); err != nil {
log.Error("Error whilst indexing: %v Error: %v", iData, err)
}
}
issueIndexerQueue = queue.CreateQueue("issue_indexer", handler, &IndexerData{})
if issueIndexerQueue == nil {
log.Fatal("Unable to create issue indexer queue")
}
default:
issueIndexerQueue = &queue.DummyQueue{}
}
// Create the Indexer
go func() {
start := time.Now()
log.Info("Initializing Issue Indexer")
log.Info("PID %d: Initializing Issue Indexer: %s", os.Getpid(), setting.Indexer.IssueType)
var populate bool
var dummyQueue bool
switch setting.Indexer.IssueType {
case "bleve":
issueIndexer := NewBleveIndexer(setting.Indexer.IssuePath)
exist, err := issueIndexer.Init()
if err != nil {
log.Fatal("Unable to initialize Bleve Issue Indexer: %v", err)
}
populate = !exist
holder.set(issueIndexer)
graceful.GetManager().RunWithShutdownFns(func(_, atTerminate func(context.Context, func())) {
issueIndexer := NewBleveIndexer(setting.Indexer.IssuePath)
exist, err := issueIndexer.Init()
if err != nil {
holder.cancel()
log.Fatal("Unable to initialize Bleve Issue Indexer: %v", err)
}
populate = !exist
holder.set(issueIndexer)
atTerminate(context.Background(), func() {
log.Debug("Closing issue indexer")
issueIndexer := holder.get()
if issueIndexer != nil {
issueIndexer.Close()
}
log.Info("PID: %d Issue Indexer closed", os.Getpid())
})
log.Debug("Created Bleve Indexer")
})
case "db":
issueIndexer := &DBIndexer{}
holder.set(issueIndexer)
dummyQueue = true
default:
holder.cancel()
log.Fatal("Unknown issue indexer type: %s", setting.Indexer.IssueType)
}
if dummyQueue {
issueIndexerQueue = &DummyQueue{}
} else {
var err error
switch setting.Indexer.IssueQueueType {
case setting.LevelQueueType:
issueIndexerQueue, err = NewLevelQueue(
holder.get(),
setting.Indexer.IssueQueueDir,
setting.Indexer.IssueQueueBatchNumber)
if err != nil {
log.Fatal(
"Unable create level queue for issue queue dir: %s batch number: %d : %v",
setting.Indexer.IssueQueueDir,
setting.Indexer.IssueQueueBatchNumber,
err)
}
case setting.ChannelQueueType:
issueIndexerQueue = NewChannelQueue(holder.get(), setting.Indexer.IssueQueueBatchNumber)
case setting.RedisQueueType:
addrs, pass, idx, err := parseConnStr(setting.Indexer.IssueQueueConnStr)
if err != nil {
log.Fatal("Unable to parse connection string for RedisQueueType: %s : %v",
setting.Indexer.IssueQueueConnStr,
err)
}
issueIndexerQueue, err = NewRedisQueue(addrs, pass, idx, holder.get(), setting.Indexer.IssueQueueBatchNumber)
if err != nil {
log.Fatal("Unable to create RedisQueue: %s : %v",
setting.Indexer.IssueQueueConnStr,
err)
}
default:
log.Fatal("Unsupported indexer queue type: %v",
setting.Indexer.IssueQueueType)
}
go func() {
err = issueIndexerQueue.Run()
if err != nil {
log.Error("issueIndexerQueue.Run: %v", err)
}
}()
}
go func() {
for data := range issueIndexerChannel {
_ = issueIndexerQueue.Push(data)
}
}()
// Start processing the queue
go graceful.GetManager().RunWithShutdownFns(issueIndexerQueue.Run)
// Populate the index
if populate {
if syncReindex {
populateIssueIndexer()
graceful.GetManager().RunWithShutdownContext(populateIssueIndexer)
} else {
go populateIssueIndexer()
go graceful.GetManager().RunWithShutdownContext(populateIssueIndexer)
}
}
waitChannel <- time.Since(start)
close(waitChannel)
}()
if syncReindex {
<-waitChannel
select {
case <-waitChannel:
case <-graceful.GetManager().IsShutdown():
}
} else if setting.Indexer.StartupTimeout > 0 {
go func() {
timeout := setting.Indexer.StartupTimeout
@ -178,7 +199,12 @@ func InitIssueIndexer(syncReindex bool) {
select {
case duration := <-waitChannel:
log.Info("Issue Indexer Initialization took %v", duration)
case <-graceful.GetManager().IsShutdown():
log.Warn("Shutdown occurred before issue index initialisation was complete")
case <-time.After(timeout):
if shutdownable, ok := issueIndexerQueue.(queue.Shutdownable); ok {
shutdownable.Terminate()
}
log.Fatal("Issue Indexer Initialization timed-out after: %v", timeout)
}
}()
@ -186,8 +212,14 @@ func InitIssueIndexer(syncReindex bool) {
}
// populateIssueIndexer populate the issue indexer with issue data
func populateIssueIndexer() {
func populateIssueIndexer(ctx context.Context) {
for page := 1; ; page++ {
select {
case <-ctx.Done():
log.Warn("Issue Indexer population shutdown before completion")
return
default:
}
repos, _, err := models.SearchRepositoryByName(&models.SearchRepoOptions{
Page: page,
PageSize: models.RepositoryListDefaultPageSize,
@ -200,10 +232,17 @@ func populateIssueIndexer() {
continue
}
if len(repos) == 0 {
log.Debug("Issue Indexer population complete")
return
}
for _, repo := range repos {
select {
case <-ctx.Done():
log.Info("Issue Indexer population shutdown before completion")
return
default:
}
UpdateRepoIndexer(repo)
}
}
@ -237,13 +276,17 @@ func UpdateIssueIndexer(issue *models.Issue) {
comments = append(comments, comment.Content)
}
}
issueIndexerChannel <- &IndexerData{
indexerData := &IndexerData{
ID: issue.ID,
RepoID: issue.RepoID,
Title: issue.Title,
Content: issue.Content,
Comments: comments,
}
log.Debug("Adding to channel: %v", indexerData)
if err := issueIndexerQueue.Push(indexerData); err != nil {
log.Error("Unable to push to issue indexer: %v: Error: %v", indexerData, err)
}
}
// DeleteRepoIssueIndexer deletes repo's all issues indexes
@ -258,17 +301,25 @@ func DeleteRepoIssueIndexer(repo *models.Repository) {
if len(ids) == 0 {
return
}
issueIndexerChannel <- &IndexerData{
indexerData := &IndexerData{
IDs: ids,
IsDelete: true,
}
if err := issueIndexerQueue.Push(indexerData); err != nil {
log.Error("Unable to push to issue indexer: %v: Error: %v", indexerData, err)
}
}
// SearchIssuesByKeyword search issue ids by keywords and repo id
func SearchIssuesByKeyword(repoIDs []int64, keyword string) ([]int64, error) {
var issueIDs []int64
res, err := holder.get().Search(keyword, repoIDs, 1000, 0)
indexer := holder.get()
if indexer == nil {
log.Error("SearchIssuesByKeyword(): unable to get indexer!")
return nil, fmt.Errorf("unable to get issue indexer")
}
res, err := indexer.Search(keyword, repoIDs, 1000, 0)
if err != nil {
return nil, err
}

4
modules/indexer/issues/indexer_test.go
View file

@ -15,6 +15,8 @@ import (
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/setting"
"gopkg.in/ini.v1"
"github.com/stretchr/testify/assert"
)
@ -24,6 +26,7 @@ func TestMain(m *testing.M) {
func TestBleveSearchIssues(t *testing.T) {
assert.NoError(t, models.PrepareTestDatabase())
setting.Cfg = ini.Empty()
tmpIndexerDir, err := ioutil.TempDir("", "issues-indexer")
if err != nil {
@ -41,6 +44,7 @@ func TestBleveSearchIssues(t *testing.T) {
}()
setting.Indexer.IssueType = "bleve"
setting.NewQueueService()
InitIssueIndexer(true)
defer func() {
indexer := holder.get()

25
modules/indexer/issues/queue.go
View file

@ -1,25 +0,0 @@
// Copyright 2018 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package issues
// Queue defines an interface to save an issue indexer queue
type Queue interface {
Run() error
Push(*IndexerData) error
}
// DummyQueue represents an empty queue
type DummyQueue struct {
}
// Run starts to run the queue
func (b *DummyQueue) Run() error {
return nil
}
// Push pushes data to indexer
func (b *DummyQueue) Push(*IndexerData) error {
return nil
}

62
modules/indexer/issues/queue_channel.go
View file

@ -1,62 +0,0 @@
// Copyright 2018 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package issues
import (
"time"
"code.gitea.io/gitea/modules/setting"
)
// ChannelQueue implements
type ChannelQueue struct {
queue chan *IndexerData
indexer Indexer
batchNumber int
}
// NewChannelQueue create a memory channel queue
func NewChannelQueue(indexer Indexer, batchNumber int) *ChannelQueue {
return &ChannelQueue{
queue: make(chan *IndexerData, setting.Indexer.UpdateQueueLength),
indexer: indexer,
batchNumber: batchNumber,
}
}
// Run starts to run the queue
func (c *ChannelQueue) Run() error {
var i int
var datas = make([]*IndexerData, 0, c.batchNumber)
for {
select {
case data := <-c.queue:
if data.IsDelete {
_ = c.indexer.Delete(data.IDs...)
continue
}
datas = append(datas, data)
if len(datas) >= c.batchNumber {
_ = c.indexer.Index(datas)
// TODO: save the point
datas = make([]*IndexerData, 0, c.batchNumber)
}
case <-time.After(time.Millisecond * 100):
i++
if i >= 3 && len(datas) > 0 {
_ = c.indexer.Index(datas)
// TODO: save the point
datas = make([]*IndexerData, 0, c.batchNumber)
}
}
}
}
// Push will push the indexer data to queue
func (c *ChannelQueue) Push(data *IndexerData) error {
c.queue <- data
return nil
}

104
modules/indexer/issues/queue_disk.go
View file

@ -1,104 +0,0 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package issues
import (
"encoding/json"
"time"
"code.gitea.io/gitea/modules/log"
"gitea.com/lunny/levelqueue"
)
var (
_ Queue = &LevelQueue{}
)
// LevelQueue implements a disk library queue
type LevelQueue struct {
indexer Indexer
queue *levelqueue.Queue
batchNumber int
}
// NewLevelQueue creates a ledis local queue
func NewLevelQueue(indexer Indexer, dataDir string, batchNumber int) (*LevelQueue, error) {
queue, err := levelqueue.Open(dataDir)
if err != nil {
return nil, err
}
return &LevelQueue{
indexer: indexer,
queue: queue,
batchNumber: batchNumber,
}, nil
}
// Run starts to run the queue
func (l *LevelQueue) Run() error {
var i int
var datas = make([]*IndexerData, 0, l.batchNumber)
for {
i++
if len(datas) > l.batchNumber || (len(datas) > 0 && i > 3) {
_ = l.indexer.Index(datas)
datas = make([]*IndexerData, 0, l.batchNumber)
i = 0
continue
}
bs, err := l.queue.RPop()
if err != nil {
if err != levelqueue.ErrNotFound {
log.Error("RPop: %v", err)
}
time.Sleep(time.Millisecond * 100)
continue
}
if len(bs) == 0 {
time.Sleep(time.Millisecond * 100)
continue
}
var data IndexerData
err = json.Unmarshal(bs, &data)
if err != nil {
log.Error("Unmarshal: %v", err)
time.Sleep(time.Millisecond * 100)
continue
}
log.Trace("LevelQueue: task found: %#v", data)
if data.IsDelete {
if data.ID > 0 {
if err = l.indexer.Delete(data.ID); err != nil {
log.Error("indexer.Delete: %v", err)
}
} else if len(data.IDs) > 0 {
if err = l.indexer.Delete(data.IDs...); err != nil {
log.Error("indexer.Delete: %v", err)
}
}
time.Sleep(time.Millisecond * 10)
continue
}
datas = append(datas, &data)
time.Sleep(time.Millisecond * 10)
}
}
// Push will push the indexer data to queue
func (l *LevelQueue) Push(data *IndexerData) error {
bs, err := json.Marshal(data)
if err != nil {
return err
}
return l.queue.LPush(bs)
}

146
modules/indexer/issues/queue_redis.go
View file

@ -1,146 +0,0 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package issues
import (
"encoding/json"
"errors"
"strconv"
"strings"
"time"
"code.gitea.io/gitea/modules/log"
"github.com/go-redis/redis"
)
var (
_ Queue = &RedisQueue{}
)
type redisClient interface {
RPush(key string, args ...interface{}) *redis.IntCmd
LPop(key string) *redis.StringCmd
Ping() *redis.StatusCmd
}
// RedisQueue redis queue
type RedisQueue struct {
client redisClient
queueName string
indexer Indexer
batchNumber int
}
func parseConnStr(connStr string) (addrs, password string, dbIdx int, err error) {
fields := strings.Fields(connStr)
for _, f := range fields {
items := strings.SplitN(f, "=", 2)
if len(items) < 2 {
continue
}
switch strings.ToLower(items[0]) {
case "addrs":
addrs = items[1]
case "password":
password = items[1]
case "db":
dbIdx, err = strconv.Atoi(items[1])
if err != nil {
return
}
}
}
return
}
// NewRedisQueue creates single redis or cluster redis queue
func NewRedisQueue(addrs string, password string, dbIdx int, indexer Indexer, batchNumber int) (*RedisQueue, error) {
dbs := strings.Split(addrs, ",")
var queue = RedisQueue{
queueName: "issue_indexer_queue",
indexer: indexer,
batchNumber: batchNumber,
}
if len(dbs) == 0 {
return nil, errors.New("no redis host found")
} else if len(dbs) == 1 {
queue.client = redis.NewClient(&redis.Options{
Addr: strings.TrimSpace(dbs[0]), // use default Addr
Password: password, // no password set
DB: dbIdx, // use default DB
})
} else {
queue.client = redis.NewClusterClient(&redis.ClusterOptions{
Addrs: dbs,
})
}
if err := queue.client.Ping().Err(); err != nil {
return nil, err
}
return &queue, nil
}
// Run runs the redis queue
func (r *RedisQueue) Run() error {
var i int
var datas = make([]*IndexerData, 0, r.batchNumber)
for {
bs, err := r.client.LPop(r.queueName).Bytes()
if err != nil && err != redis.Nil {
log.Error("LPop faile: %v", err)
time.Sleep(time.Millisecond * 100)
continue
}
i++
if len(datas) > r.batchNumber || (len(datas) > 0 && i > 3) {
_ = r.indexer.Index(datas)
datas = make([]*IndexerData, 0, r.batchNumber)
i = 0
}
if len(bs) == 0 {
time.Sleep(time.Millisecond * 100)
continue
}
var data IndexerData
err = json.Unmarshal(bs, &data)
if err != nil {
log.Error("Unmarshal: %v", err)
time.Sleep(time.Millisecond * 100)
continue
}
log.Trace("RedisQueue: task found: %#v", data)
if data.IsDelete {
if data.ID > 0 {
if err = r.indexer.Delete(data.ID); err != nil {
log.Error("indexer.Delete: %v", err)
}
} else if len(data.IDs) > 0 {
if err = r.indexer.Delete(data.IDs...); err != nil {
log.Error("indexer.Delete: %v", err)
}
}
time.Sleep(time.Millisecond * 100)
continue
}
datas = append(datas, &data)
time.Sleep(time.Millisecond * 100)
}
}
// Push implements Queue
func (r *RedisQueue) Push(data *IndexerData) error {
bs, err := json.Marshal(data)
if err != nil {
return err
}
return r.client.RPush(r.queueName, bs).Err()
}

270
modules/queue/manager.go Normal file
View file

@ -0,0 +1,270 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"encoding/json"
"fmt"
"reflect"
"sort"
"sync"
"time"
"code.gitea.io/gitea/modules/log"
)
var manager *Manager
// Manager is a queue manager
type Manager struct {
mutex sync.Mutex
counter int64
Queues map[int64]*ManagedQueue
}
// ManagedQueue represents a working queue inheriting from Gitea.
type ManagedQueue struct {
mutex sync.Mutex
QID int64
Queue Queue
Type Type
Name string
Configuration interface{}
ExemplarType string
Pool ManagedPool
counter int64
PoolWorkers map[int64]*PoolWorkers
}
// ManagedPool is a simple interface to get certain details from a worker pool
type ManagedPool interface {
AddWorkers(number int, timeout time.Duration) context.CancelFunc
NumberOfWorkers() int
MaxNumberOfWorkers() int
SetMaxNumberOfWorkers(int)
BoostTimeout() time.Duration
BlockTimeout() time.Duration
BoostWorkers() int
SetSettings(maxNumberOfWorkers, boostWorkers int, timeout time.Duration)
}
// ManagedQueueList implements the sort.Interface
type ManagedQueueList []*ManagedQueue
// PoolWorkers represents a working queue inheriting from Gitea.
type PoolWorkers struct {
PID int64
Workers int
Start time.Time
Timeout time.Time
HasTimeout bool
Cancel context.CancelFunc
}
// PoolWorkersList implements the sort.Interface
type PoolWorkersList []*PoolWorkers
func init() {
_ = GetManager()
}
// GetManager returns a Manager and initializes one as singleton if there's none yet
func GetManager() *Manager {
if manager == nil {
manager = &Manager{
Queues: make(map[int64]*ManagedQueue),
}
}
return manager
}
// Add adds a queue to this manager
func (m *Manager) Add(queue Queue,
t Type,
configuration,
exemplar interface{},
pool ManagedPool) int64 {
cfg, _ := json.Marshal(configuration)
mq := &ManagedQueue{
Queue: queue,
Type: t,
Configuration: string(cfg),
ExemplarType: reflect.TypeOf(exemplar).String(),
PoolWorkers: make(map[int64]*PoolWorkers),
Pool: pool,
}
m.mutex.Lock()
m.counter++
mq.QID = m.counter
mq.Name = fmt.Sprintf("queue-%d", mq.QID)
if named, ok := queue.(Named); ok {
mq.Name = named.Name()
}
m.Queues[mq.QID] = mq
m.mutex.Unlock()
log.Trace("Queue Manager registered: %s (QID: %d)", mq.Name, mq.QID)
return mq.QID
}
// Remove a queue from the Manager
func (m *Manager) Remove(qid int64) {
m.mutex.Lock()
delete(m.Queues, qid)
m.mutex.Unlock()
log.Trace("Queue Manager removed: QID: %d", qid)
}
// GetManagedQueue by qid
func (m *Manager) GetManagedQueue(qid int64) *ManagedQueue {
m.mutex.Lock()
defer m.mutex.Unlock()
return m.Queues[qid]
}
// ManagedQueues returns the managed queues
func (m *Manager) ManagedQueues() []*ManagedQueue {
m.mutex.Lock()
mqs := make([]*ManagedQueue, 0, len(m.Queues))
for _, mq := range m.Queues {
mqs = append(mqs, mq)
}
m.mutex.Unlock()
sort.Sort(ManagedQueueList(mqs))
return mqs
}
// Workers returns the poolworkers
func (q *ManagedQueue) Workers() []*PoolWorkers {
q.mutex.Lock()
workers := make([]*PoolWorkers, 0, len(q.PoolWorkers))
for _, worker := range q.PoolWorkers {
workers = append(workers, worker)
}
q.mutex.Unlock()
sort.Sort(PoolWorkersList(workers))
return workers
}
// RegisterWorkers registers workers to this queue
func (q *ManagedQueue) RegisterWorkers(number int, start time.Time, hasTimeout bool, timeout time.Time, cancel context.CancelFunc) int64 {
q.mutex.Lock()
defer q.mutex.Unlock()
q.counter++
q.PoolWorkers[q.counter] = &PoolWorkers{
PID: q.counter,
Workers: number,
Start: start,
Timeout: timeout,
HasTimeout: hasTimeout,
Cancel: cancel,
}
return q.counter
}
// CancelWorkers cancels pooled workers with pid
func (q *ManagedQueue) CancelWorkers(pid int64) {
q.mutex.Lock()
pw, ok := q.PoolWorkers[pid]
q.mutex.Unlock()
if !ok {
return
}
pw.Cancel()
}
// RemoveWorkers deletes pooled workers with pid
func (q *ManagedQueue) RemoveWorkers(pid int64) {
q.mutex.Lock()
pw, ok := q.PoolWorkers[pid]
delete(q.PoolWorkers, pid)
q.mutex.Unlock()
if ok && pw.Cancel != nil {
pw.Cancel()
}
}
// AddWorkers adds workers to the queue if it has registered an add worker function
func (q *ManagedQueue) AddWorkers(number int, timeout time.Duration) context.CancelFunc {
if q.Pool != nil {
// the cancel will be added to the pool workers description above
return q.Pool.AddWorkers(number, timeout)
}
return nil
}
// NumberOfWorkers returns the number of workers in the queue
func (q *ManagedQueue) NumberOfWorkers() int {
if q.Pool != nil {
return q.Pool.NumberOfWorkers()
}
return -1
}
// MaxNumberOfWorkers returns the maximum number of workers for the pool
func (q *ManagedQueue) MaxNumberOfWorkers() int {
if q.Pool != nil {
return q.Pool.MaxNumberOfWorkers()
}
return 0
}
// BoostWorkers returns the number of workers for a boost
func (q *ManagedQueue) BoostWorkers() int {
if q.Pool != nil {
return q.Pool.BoostWorkers()
}
return -1
}
// BoostTimeout returns the timeout of the next boost
func (q *ManagedQueue) BoostTimeout() time.Duration {
if q.Pool != nil {
return q.Pool.BoostTimeout()
}
return 0
}
// BlockTimeout returns the timeout til the next boost
func (q *ManagedQueue) BlockTimeout() time.Duration {
if q.Pool != nil {
return q.Pool.BlockTimeout()
}
return 0
}
// SetSettings sets the setable boost values
func (q *ManagedQueue) SetSettings(maxNumberOfWorkers, boostWorkers int, timeout time.Duration) {
if q.Pool != nil {
q.Pool.SetSettings(maxNumberOfWorkers, boostWorkers, timeout)
}
}
func (l ManagedQueueList) Len() int {
return len(l)
}
func (l ManagedQueueList) Less(i, j int) bool {
return l[i].Name < l[j].Name
}
func (l ManagedQueueList) Swap(i, j int) {
l[i], l[j] = l[j], l[i]
}
func (l PoolWorkersList) Len() int {
return len(l)
}
func (l PoolWorkersList) Less(i, j int) bool {
return l[i].Start.Before(l[j].Start)
}
func (l PoolWorkersList) Swap(i, j int) {
l[i], l[j] = l[j], l[i]
}

133
modules/queue/queue.go Normal file
View file

@ -0,0 +1,133 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"encoding/json"
"fmt"
"reflect"
)
// ErrInvalidConfiguration is called when there is invalid configuration for a queue
type ErrInvalidConfiguration struct {
cfg interface{}
err error
}
func (err ErrInvalidConfiguration) Error() string {
if err.err != nil {
return fmt.Sprintf("Invalid Configuration Argument: %v: Error: %v", err.cfg, err.err)
}
return fmt.Sprintf("Invalid Configuration Argument: %v", err.cfg)
}
// IsErrInvalidConfiguration checks if an error is an ErrInvalidConfiguration
func IsErrInvalidConfiguration(err error) bool {
_, ok := err.(ErrInvalidConfiguration)
return ok
}
// Type is a type of Queue
type Type string
// Data defines an type of queuable data
type Data interface{}
// HandlerFunc is a function that takes a variable amount of data and processes it
type HandlerFunc func(...Data)
// NewQueueFunc is a function that creates a queue
type NewQueueFunc func(handler HandlerFunc, config interface{}, exemplar interface{}) (Queue, error)
// Shutdownable represents a queue that can be shutdown
type Shutdownable interface {
Shutdown()
Terminate()
}
// Named represents a queue with a name
type Named interface {
Name() string
}
// Queue defines an interface to save an issue indexer queue
type Queue interface {
Run(atShutdown, atTerminate func(context.Context, func()))
Push(Data) error
}
// DummyQueueType is the type for the dummy queue
const DummyQueueType Type = "dummy"
// NewDummyQueue creates a new DummyQueue
func NewDummyQueue(handler HandlerFunc, opts, exemplar interface{}) (Queue, error) {
return &DummyQueue{}, nil
}
// DummyQueue represents an empty queue
type DummyQueue struct {
}
// Run starts to run the queue
func (b *DummyQueue) Run(_, _ func(context.Context, func())) {}
// Push pushes data to the queue
func (b *DummyQueue) Push(Data) error {
return nil
}
func toConfig(exemplar, cfg interface{}) (interface{}, error) {
if reflect.TypeOf(cfg).AssignableTo(reflect.TypeOf(exemplar)) {
return cfg, nil
}
configBytes, ok := cfg.([]byte)
if !ok {
configStr, ok := cfg.(string)
if !ok {
return nil, ErrInvalidConfiguration{cfg: cfg}
}
configBytes = []byte(configStr)
}
newVal := reflect.New(reflect.TypeOf(exemplar))
if err := json.Unmarshal(configBytes, newVal.Interface()); err != nil {
return nil, ErrInvalidConfiguration{cfg: cfg, err: err}
}
return newVal.Elem().Interface(), nil
}
var queuesMap = map[Type]NewQueueFunc{DummyQueueType: NewDummyQueue}
// RegisteredTypes provides the list of requested types of queues
func RegisteredTypes() []Type {
types := make([]Type, len(queuesMap))
i := 0
for key := range queuesMap {
types[i] = key
i++
}
return types
}
// RegisteredTypesAsString provides the list of requested types of queues
func RegisteredTypesAsString() []string {
types := make([]string, len(queuesMap))
i := 0
for key := range queuesMap {
types[i] = string(key)
i++
}
return types
}
// NewQueue takes a queue Type and HandlerFunc some options and possibly an exemplar and returns a Queue or an error
func NewQueue(queueType Type, handlerFunc HandlerFunc, opts, exemplar interface{}) (Queue, error) {
newFn, ok := queuesMap[queueType]
if !ok {
return nil, fmt.Errorf("Unsupported queue type: %v", queueType)
}
return newFn(handlerFunc, opts, exemplar)
}

106
modules/queue/queue_channel.go Normal file
View file

@ -0,0 +1,106 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"fmt"
"reflect"
"time"
"code.gitea.io/gitea/modules/log"
)
// ChannelQueueType is the type for channel queue
const ChannelQueueType Type = "channel"
// ChannelQueueConfiguration is the configuration for a ChannelQueue
type ChannelQueueConfiguration struct {
QueueLength int
BatchLength int
Workers int
MaxWorkers int
BlockTimeout time.Duration
BoostTimeout time.Duration
BoostWorkers int
Name string
}
// ChannelQueue implements
type ChannelQueue struct {
pool *WorkerPool
exemplar interface{}
workers int
name string
}
// NewChannelQueue create a memory channel queue
func NewChannelQueue(handle HandlerFunc, cfg, exemplar interface{}) (Queue, error) {
configInterface, err := toConfig(ChannelQueueConfiguration{}, cfg)
if err != nil {
return nil, err
}
config := configInterface.(ChannelQueueConfiguration)
if config.BatchLength == 0 {
config.BatchLength = 1
}
dataChan := make(chan Data, config.QueueLength)
ctx, cancel := context.WithCancel(context.Background())
queue := &ChannelQueue{
pool: &WorkerPool{
baseCtx: ctx,
cancel: cancel,
batchLength: config.BatchLength,
handle: handle,
dataChan: dataChan,
blockTimeout: config.BlockTimeout,
boostTimeout: config.BoostTimeout,
boostWorkers: config.BoostWorkers,
maxNumberOfWorkers: config.MaxWorkers,
},
exemplar: exemplar,
workers: config.Workers,
name: config.Name,
}
queue.pool.qid = GetManager().Add(queue, ChannelQueueType, config, exemplar, queue.pool)
return queue, nil
}
// Run starts to run the queue
func (c *ChannelQueue) Run(atShutdown, atTerminate func(context.Context, func())) {
atShutdown(context.Background(), func() {
log.Warn("ChannelQueue: %s is not shutdownable!", c.name)
})
atTerminate(context.Background(), func() {
log.Warn("ChannelQueue: %s is not terminatable!", c.name)
})
go func() {
_ = c.pool.AddWorkers(c.workers, 0)
}()
}
// Push will push data into the queue
func (c *ChannelQueue) Push(data Data) error {
if c.exemplar != nil {
// Assert data is of same type as r.exemplar
t := reflect.TypeOf(data)
exemplarType := reflect.TypeOf(c.exemplar)
if !t.AssignableTo(exemplarType) || data == nil {
return fmt.Errorf("Unable to assign data: %v to same type as exemplar: %v in queue: %s", data, c.exemplar, c.name)
}
}
c.pool.Push(data)
return nil
}
// Name returns the name of this queue
func (c *ChannelQueue) Name() string {
return c.name
}
func init() {
queuesMap[ChannelQueueType] = NewChannelQueue
}

91
modules/queue/queue_channel_test.go Normal file
View file

@ -0,0 +1,91 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestChannelQueue(t *testing.T) {
handleChan := make(chan *testData)
handle := func(data ...Data) {
for _, datum := range data {
testDatum := datum.(*testData)
handleChan <- testDatum
}
}
nilFn := func(_ context.Context, _ func()) {}
queue, err := NewChannelQueue(handle,
ChannelQueueConfiguration{
QueueLength: 20,
Workers: 1,
MaxWorkers: 10,
BlockTimeout: 1 * time.Second,
BoostTimeout: 5 * time.Minute,
BoostWorkers: 5,
}, &testData{})
assert.NoError(t, err)
go queue.Run(nilFn, nilFn)
test1 := testData{"A", 1}
go queue.Push(&test1)
result1 := <-handleChan
assert.Equal(t, test1.TestString, result1.TestString)
assert.Equal(t, test1.TestInt, result1.TestInt)
err = queue.Push(test1)
assert.Error(t, err)
}
func TestChannelQueue_Batch(t *testing.T) {
handleChan := make(chan *testData)
handle := func(data ...Data) {
assert.True(t, len(data) == 2)
for _, datum := range data {
testDatum := datum.(*testData)
handleChan <- testDatum
}
}
nilFn := func(_ context.Context, _ func()) {}
queue, err := NewChannelQueue(handle,
ChannelQueueConfiguration{
QueueLength: 20,
BatchLength: 2,
Workers: 1,
MaxWorkers: 10,
BlockTimeout: 1 * time.Second,
BoostTimeout: 5 * time.Minute,
BoostWorkers: 5,
}, &testData{})
assert.NoError(t, err)
go queue.Run(nilFn, nilFn)
test1 := testData{"A", 1}
test2 := testData{"B", 2}
queue.Push(&test1)
go queue.Push(&test2)
result1 := <-handleChan
assert.Equal(t, test1.TestString, result1.TestString)
assert.Equal(t, test1.TestInt, result1.TestInt)
result2 := <-handleChan
assert.Equal(t, test2.TestString, result2.TestString)
assert.Equal(t, test2.TestInt, result2.TestInt)
err = queue.Push(test1)
assert.Error(t, err)
}

213
modules/queue/queue_disk.go Normal file
View file

@ -0,0 +1,213 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"encoding/json"
"fmt"
"reflect"
"sync"
"time"
"code.gitea.io/gitea/modules/log"
"gitea.com/lunny/levelqueue"
)
// LevelQueueType is the type for level queue
const LevelQueueType Type = "level"
// LevelQueueConfiguration is the configuration for a LevelQueue
type LevelQueueConfiguration struct {
DataDir string
QueueLength int
BatchLength int
Workers int
MaxWorkers int
BlockTimeout time.Duration
BoostTimeout time.Duration
BoostWorkers int
Name string
}
// LevelQueue implements a disk library queue
type LevelQueue struct {
pool *WorkerPool
queue *levelqueue.Queue
closed chan struct{}
terminated chan struct{}
lock sync.Mutex
exemplar interface{}
workers int
name string
}
// NewLevelQueue creates a ledis local queue
func NewLevelQueue(handle HandlerFunc, cfg, exemplar interface{}) (Queue, error) {
configInterface, err := toConfig(LevelQueueConfiguration{}, cfg)
if err != nil {
return nil, err
}
config := configInterface.(LevelQueueConfiguration)
internal, err := levelqueue.Open(config.DataDir)
if err != nil {
return nil, err
}
dataChan := make(chan Data, config.QueueLength)
ctx, cancel := context.WithCancel(context.Background())
queue := &LevelQueue{
pool: &WorkerPool{
baseCtx: ctx,
cancel: cancel,
batchLength: config.BatchLength,
handle: handle,
dataChan: dataChan,
blockTimeout: config.BlockTimeout,
boostTimeout: config.BoostTimeout,
boostWorkers: config.BoostWorkers,
maxNumberOfWorkers: config.MaxWorkers,
},
queue: internal,
exemplar: exemplar,
closed: make(chan struct{}),
terminated: make(chan struct{}),
workers: config.Workers,
name: config.Name,
}
queue.pool.qid = GetManager().Add(queue, LevelQueueType, config, exemplar, queue.pool)
return queue, nil
}
// Run starts to run the queue
func (l *LevelQueue) Run(atShutdown, atTerminate func(context.Context, func())) {
atShutdown(context.Background(), l.Shutdown)
atTerminate(context.Background(), l.Terminate)
go func() {
_ = l.pool.AddWorkers(l.workers, 0)
}()
go l.readToChan()
log.Trace("LevelQueue: %s Waiting til closed", l.name)
<-l.closed
log.Trace("LevelQueue: %s Waiting til done", l.name)
l.pool.Wait()
log.Trace("LevelQueue: %s Waiting til cleaned", l.name)
ctx, cancel := context.WithCancel(context.Background())
atTerminate(ctx, cancel)
l.pool.CleanUp(ctx)
cancel()
log.Trace("LevelQueue: %s Cleaned", l.name)
}
func (l *LevelQueue) readToChan() {
for {
select {
case <-l.closed:
// tell the pool to shutdown.
l.pool.cancel()
return
default:
bs, err := l.queue.RPop()
if err != nil {
if err != levelqueue.ErrNotFound {
log.Error("LevelQueue: %s Error on RPop: %v", l.name, err)
}
time.Sleep(time.Millisecond * 100)
continue
}
if len(bs) == 0 {
time.Sleep(time.Millisecond * 100)
continue
}
var data Data
if l.exemplar != nil {
t := reflect.TypeOf(l.exemplar)
n := reflect.New(t)
ne := n.Elem()
err = json.Unmarshal(bs, ne.Addr().Interface())
data = ne.Interface().(Data)
} else {
err = json.Unmarshal(bs, &data)
}
if err != nil {
log.Error("LevelQueue: %s Failed to unmarshal with error: %v", l.name, err)
time.Sleep(time.Millisecond * 100)
continue
}
log.Trace("LevelQueue %s: Task found: %#v", l.name, data)
l.pool.Push(data)
}
}
}
// Push will push the indexer data to queue
func (l *LevelQueue) Push(data Data) error {
if l.exemplar != nil {
// Assert data is of same type as r.exemplar
value := reflect.ValueOf(data)
t := value.Type()
exemplarType := reflect.ValueOf(l.exemplar).Type()
if !t.AssignableTo(exemplarType) || data == nil {
return fmt.Errorf("Unable to assign data: %v to same type as exemplar: %v in %s", data, l.exemplar, l.name)
}
}
bs, err := json.Marshal(data)
if err != nil {
return err
}
return l.queue.LPush(bs)
}
// Shutdown this queue and stop processing
func (l *LevelQueue) Shutdown() {
l.lock.Lock()
defer l.lock.Unlock()
log.Trace("LevelQueue: %s Shutdown", l.name)
select {
case <-l.closed:
default:
close(l.closed)
}
}
// Terminate this queue and close the queue
func (l *LevelQueue) Terminate() {
log.Trace("LevelQueue: %s Terminating", l.name)
l.Shutdown()
l.lock.Lock()
select {
case <-l.terminated:
l.lock.Unlock()
default:
close(l.terminated)
l.lock.Unlock()
if err := l.queue.Close(); err != nil && err.Error() != "leveldb: closed" {
log.Error("Error whilst closing internal queue in %s: %v", l.name, err)
}
}
}
// Name returns the name of this queue
func (l *LevelQueue) Name() string {
return l.name
}
func init() {
queuesMap[LevelQueueType] = NewLevelQueue
}

193
modules/queue/queue_disk_channel.go Normal file
View file

@ -0,0 +1,193 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"time"
"code.gitea.io/gitea/modules/log"
)
// PersistableChannelQueueType is the type for persistable queue
const PersistableChannelQueueType Type = "persistable-channel"
// PersistableChannelQueueConfiguration is the configuration for a PersistableChannelQueue
type PersistableChannelQueueConfiguration struct {
Name string
DataDir string
BatchLength int
QueueLength int
Timeout time.Duration
MaxAttempts int
Workers int
MaxWorkers int
BlockTimeout time.Duration
BoostTimeout time.Duration
BoostWorkers int
}
// PersistableChannelQueue wraps a channel queue and level queue together
type PersistableChannelQueue struct {
*ChannelQueue
delayedStarter
closed chan struct{}
}
// NewPersistableChannelQueue creates a wrapped batched channel queue with persistable level queue backend when shutting down
// This differs from a wrapped queue in that the persistent queue is only used to persist at shutdown/terminate
func NewPersistableChannelQueue(handle HandlerFunc, cfg, exemplar interface{}) (Queue, error) {
configInterface, err := toConfig(PersistableChannelQueueConfiguration{}, cfg)
if err != nil {
return nil, err
}
config := configInterface.(PersistableChannelQueueConfiguration)
channelQueue, err := NewChannelQueue(handle, ChannelQueueConfiguration{
QueueLength: config.QueueLength,
BatchLength: config.BatchLength,
Workers: config.Workers,
MaxWorkers: config.MaxWorkers,
BlockTimeout: config.BlockTimeout,
BoostTimeout: config.BoostTimeout,
BoostWorkers: config.BoostWorkers,
Name: config.Name + "-channel",
}, exemplar)
if err != nil {
return nil, err
}
// the level backend only needs temporary workers to catch up with the previously dropped work
levelCfg := LevelQueueConfiguration{
DataDir: config.DataDir,
QueueLength: config.QueueLength,
BatchLength: config.BatchLength,
Workers: 1,
MaxWorkers: 6,
BlockTimeout: 1 * time.Second,
BoostTimeout: 5 * time.Minute,
BoostWorkers: 5,
Name: config.Name + "-level",
}
levelQueue, err := NewLevelQueue(handle, levelCfg, exemplar)
if err == nil {
queue := &PersistableChannelQueue{
ChannelQueue: channelQueue.(*ChannelQueue),
delayedStarter: delayedStarter{
internal: levelQueue.(*LevelQueue),
name: config.Name,
},
closed: make(chan struct{}),
}
_ = GetManager().Add(queue, PersistableChannelQueueType, config, exemplar, nil)
return queue, nil
}
if IsErrInvalidConfiguration(err) {
// Retrying ain't gonna make this any better...
return nil, ErrInvalidConfiguration{cfg: cfg}
}
queue := &PersistableChannelQueue{
ChannelQueue: channelQueue.(*ChannelQueue),
delayedStarter: delayedStarter{
cfg: levelCfg,
underlying: LevelQueueType,
timeout: config.Timeout,
maxAttempts: config.MaxAttempts,
name: config.Name,
},
closed: make(chan struct{}),
}
_ = GetManager().Add(queue, PersistableChannelQueueType, config, exemplar, nil)
return queue, nil
}
// Name returns the name of this queue
func (p *PersistableChannelQueue) Name() string {
return p.delayedStarter.name
}
// Push will push the indexer data to queue
func (p *PersistableChannelQueue) Push(data Data) error {
select {
case <-p.closed:
return p.internal.Push(data)
default:
return p.ChannelQueue.Push(data)
}
}
// Run starts to run the queue
func (p *PersistableChannelQueue) Run(atShutdown, atTerminate func(context.Context, func())) {
p.lock.Lock()
if p.internal == nil {
err := p.setInternal(atShutdown, p.ChannelQueue.pool.handle, p.exemplar)
p.lock.Unlock()
if err != nil {
log.Fatal("Unable to create internal queue for %s Error: %v", p.Name(), err)
return
}
} else {
p.lock.Unlock()
}
atShutdown(context.Background(), p.Shutdown)
atTerminate(context.Background(), p.Terminate)
// Just run the level queue - we shut it down later
go p.internal.Run(func(_ context.Context, _ func()) {}, func(_ context.Context, _ func()) {})
go func() {
_ = p.ChannelQueue.pool.AddWorkers(p.workers, 0)
}()
log.Trace("PersistableChannelQueue: %s Waiting til closed", p.delayedStarter.name)
<-p.closed
log.Trace("PersistableChannelQueue: %s Cancelling pools", p.delayedStarter.name)
p.ChannelQueue.pool.cancel()
p.internal.(*LevelQueue).pool.cancel()
log.Trace("PersistableChannelQueue: %s Waiting til done", p.delayedStarter.name)
p.ChannelQueue.pool.Wait()
p.internal.(*LevelQueue).pool.Wait()
// Redirect all remaining data in the chan to the internal channel
go func() {
log.Trace("PersistableChannelQueue: %s Redirecting remaining data", p.delayedStarter.name)
for data := range p.ChannelQueue.pool.dataChan {
_ = p.internal.Push(data)
}
log.Trace("PersistableChannelQueue: %s Done Redirecting remaining data", p.delayedStarter.name)
}()
log.Trace("PersistableChannelQueue: %s Done main loop", p.delayedStarter.name)
}
// Shutdown processing this queue
func (p *PersistableChannelQueue) Shutdown() {
log.Trace("PersistableChannelQueue: %s Shutdown", p.delayedStarter.name)
select {
case <-p.closed:
default:
p.lock.Lock()
defer p.lock.Unlock()
if p.internal != nil {
p.internal.(*LevelQueue).Shutdown()
}
close(p.closed)
}
}
// Terminate this queue and close the queue
func (p *PersistableChannelQueue) Terminate() {
log.Trace("PersistableChannelQueue: %s Terminating", p.delayedStarter.name)
p.Shutdown()
p.lock.Lock()
defer p.lock.Unlock()
if p.internal != nil {
p.internal.(*LevelQueue).Terminate()
}
}
func init() {
queuesMap[PersistableChannelQueueType] = NewPersistableChannelQueue
}

117
modules/queue/queue_disk_channel_test.go Normal file
View file

@ -0,0 +1,117 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"io/ioutil"
"os"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestPersistableChannelQueue(t *testing.T) {
handleChan := make(chan *testData)
handle := func(data ...Data) {
assert.True(t, len(data) == 2)
for _, datum := range data {
testDatum := datum.(*testData)
handleChan <- testDatum
}
}
queueShutdown := []func(){}
queueTerminate := []func(){}
tmpDir, err := ioutil.TempDir("", "persistable-channel-queue-test-data")
assert.NoError(t, err)
defer os.RemoveAll(tmpDir)
queue, err := NewPersistableChannelQueue(handle, PersistableChannelQueueConfiguration{
DataDir: tmpDir,
BatchLength: 2,
QueueLength: 20,
Workers: 1,
MaxWorkers: 10,
}, &testData{})
assert.NoError(t, err)
go queue.Run(func(_ context.Context, shutdown func()) {
queueShutdown = append(queueShutdown, shutdown)
}, func(_ context.Context, terminate func()) {
queueTerminate = append(queueTerminate, terminate)
})
test1 := testData{"A", 1}
test2 := testData{"B", 2}
err = queue.Push(&test1)
assert.NoError(t, err)
go func() {
err = queue.Push(&test2)
assert.NoError(t, err)
}()
result1 := <-handleChan
assert.Equal(t, test1.TestString, result1.TestString)
assert.Equal(t, test1.TestInt, result1.TestInt)
result2 := <-handleChan
assert.Equal(t, test2.TestString, result2.TestString)
assert.Equal(t, test2.TestInt, result2.TestInt)
err = queue.Push(test1)
assert.Error(t, err)
for _, callback := range queueShutdown {
callback()
}
time.Sleep(200 * time.Millisecond)
err = queue.Push(&test1)
assert.NoError(t, err)
err = queue.Push(&test2)
assert.NoError(t, err)
select {
case <-handleChan:
assert.Fail(t, "Handler processing should have stopped")
default:
}
for _, callback := range queueTerminate {
callback()
}
// Reopen queue
queue, err = NewPersistableChannelQueue(handle, PersistableChannelQueueConfiguration{
DataDir: tmpDir,
BatchLength: 2,
QueueLength: 20,
Workers: 1,
MaxWorkers: 10,
}, &testData{})
assert.NoError(t, err)
go queue.Run(func(_ context.Context, shutdown func()) {
queueShutdown = append(queueShutdown, shutdown)
}, func(_ context.Context, terminate func()) {
queueTerminate = append(queueTerminate, terminate)
})
result3 := <-handleChan
assert.Equal(t, test1.TestString, result3.TestString)
assert.Equal(t, test1.TestInt, result3.TestInt)
result4 := <-handleChan
assert.Equal(t, test2.TestString, result4.TestString)
assert.Equal(t, test2.TestInt, result4.TestInt)
for _, callback := range queueShutdown {
callback()
}
for _, callback := range queueTerminate {
callback()
}
}

126
modules/queue/queue_disk_test.go Normal file
View file

@ -0,0 +1,126 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"io/ioutil"
"os"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestLevelQueue(t *testing.T) {
handleChan := make(chan *testData)
handle := func(data ...Data) {
assert.True(t, len(data) == 2)
for _, datum := range data {
testDatum := datum.(*testData)
handleChan <- testDatum
}
}
queueShutdown := []func(){}
queueTerminate := []func(){}
tmpDir, err := ioutil.TempDir("", "level-queue-test-data")
assert.NoError(t, err)
defer os.RemoveAll(tmpDir)
queue, err := NewLevelQueue(handle, LevelQueueConfiguration{
DataDir: tmpDir,
BatchLength: 2,
Workers: 1,
MaxWorkers: 10,
QueueLength: 20,
BlockTimeout: 1 * time.Second,
BoostTimeout: 5 * time.Minute,
BoostWorkers: 5,
}, &testData{})
assert.NoError(t, err)
go queue.Run(func(_ context.Context, shutdown func()) {
queueShutdown = append(queueShutdown, shutdown)
}, func(_ context.Context, terminate func()) {
queueTerminate = append(queueTerminate, terminate)
})
test1 := testData{"A", 1}
test2 := testData{"B", 2}
err = queue.Push(&test1)
assert.NoError(t, err)
go func() {
err = queue.Push(&test2)
assert.NoError(t, err)
}()
result1 := <-handleChan
assert.Equal(t, test1.TestString, result1.TestString)
assert.Equal(t, test1.TestInt, result1.TestInt)
result2 := <-handleChan
assert.Equal(t, test2.TestString, result2.TestString)
assert.Equal(t, test2.TestInt, result2.TestInt)
err = queue.Push(test1)
assert.Error(t, err)
for _, callback := range queueShutdown {
callback()
}
time.Sleep(200 * time.Millisecond)
err = queue.Push(&test1)
assert.NoError(t, err)
err = queue.Push(&test2)
assert.NoError(t, err)
select {
case <-handleChan:
assert.Fail(t, "Handler processing should have stopped")
default:
}
for _, callback := range queueTerminate {
callback()
}
// Reopen queue
queue, err = NewWrappedQueue(handle,
WrappedQueueConfiguration{
Underlying: LevelQueueType,
Config: LevelQueueConfiguration{
DataDir: tmpDir,
BatchLength: 2,
Workers: 1,
MaxWorkers: 10,
QueueLength: 20,
BlockTimeout: 1 * time.Second,
BoostTimeout: 5 * time.Minute,
BoostWorkers: 5,
},
}, &testData{})
assert.NoError(t, err)
go queue.Run(func(_ context.Context, shutdown func()) {
queueShutdown = append(queueShutdown, shutdown)
}, func(_ context.Context, terminate func()) {
queueTerminate = append(queueTerminate, terminate)
})
result3 := <-handleChan
assert.Equal(t, test1.TestString, result3.TestString)
assert.Equal(t, test1.TestInt, result3.TestInt)
result4 := <-handleChan
assert.Equal(t, test2.TestString, result4.TestString)
assert.Equal(t, test2.TestInt, result4.TestInt)
for _, callback := range queueShutdown {
callback()
}
for _, callback := range queueTerminate {
callback()
}
}

234
modules/queue/queue_redis.go Normal file
View file

@ -0,0 +1,234 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"encoding/json"
"errors"
"fmt"
"reflect"
"strings"
"sync"
"time"
"code.gitea.io/gitea/modules/log"
"github.com/go-redis/redis"
)
// RedisQueueType is the type for redis queue
const RedisQueueType Type = "redis"
type redisClient interface {
RPush(key string, args ...interface{}) *redis.IntCmd
LPop(key string) *redis.StringCmd
Ping() *redis.StatusCmd
Close() error
}
// RedisQueue redis queue
type RedisQueue struct {
pool *WorkerPool
client redisClient
queueName string
closed chan struct{}
terminated chan struct{}
exemplar interface{}
workers int
name string
lock sync.Mutex
}
// RedisQueueConfiguration is the configuration for the redis queue
type RedisQueueConfiguration struct {
Network string
Addresses string
Password string
DBIndex int
BatchLength int
QueueLength int
QueueName string
Workers int
MaxWorkers int
BlockTimeout time.Duration
BoostTimeout time.Duration
BoostWorkers int
Name string
}
// NewRedisQueue creates single redis or cluster redis queue
func NewRedisQueue(handle HandlerFunc, cfg, exemplar interface{}) (Queue, error) {
configInterface, err := toConfig(RedisQueueConfiguration{}, cfg)
if err != nil {
return nil, err
}
config := configInterface.(RedisQueueConfiguration)
dbs := strings.Split(config.Addresses, ",")
dataChan := make(chan Data, config.QueueLength)
ctx, cancel := context.WithCancel(context.Background())
var queue = &RedisQueue{
pool: &WorkerPool{
baseCtx: ctx,
cancel: cancel,
batchLength: config.BatchLength,
handle: handle,
dataChan: dataChan,
blockTimeout: config.BlockTimeout,
boostTimeout: config.BoostTimeout,
boostWorkers: config.BoostWorkers,
maxNumberOfWorkers: config.MaxWorkers,
},
queueName: config.QueueName,
exemplar: exemplar,
closed: make(chan struct{}),
workers: config.Workers,
name: config.Name,
}
if len(dbs) == 0 {
return nil, errors.New("no redis host specified")
} else if len(dbs) == 1 {
queue.client = redis.NewClient(&redis.Options{
Network: config.Network,
Addr: strings.TrimSpace(dbs[0]), // use default Addr
Password: config.Password, // no password set
DB: config.DBIndex, // use default DB
})
} else {
queue.client = redis.NewClusterClient(&redis.ClusterOptions{
Addrs: dbs,
})
}
if err := queue.client.Ping().Err(); err != nil {
return nil, err
}
queue.pool.qid = GetManager().Add(queue, RedisQueueType, config, exemplar, queue.pool)
return queue, nil
}
// Run runs the redis queue
func (r *RedisQueue) Run(atShutdown, atTerminate func(context.Context, func())) {
atShutdown(context.Background(), r.Shutdown)
atTerminate(context.Background(), r.Terminate)
go func() {
_ = r.pool.AddWorkers(r.workers, 0)
}()
go r.readToChan()
log.Trace("RedisQueue: %s Waiting til closed", r.name)
<-r.closed
log.Trace("RedisQueue: %s Waiting til done", r.name)
r.pool.Wait()
log.Trace("RedisQueue: %s Waiting til cleaned", r.name)
ctx, cancel := context.WithCancel(context.Background())
atTerminate(ctx, cancel)
r.pool.CleanUp(ctx)
cancel()
}
func (r *RedisQueue) readToChan() {
for {
select {
case <-r.closed:
// tell the pool to shutdown
r.pool.cancel()
return
default:
bs, err := r.client.LPop(r.queueName).Bytes()
if err != nil && err != redis.Nil {
log.Error("RedisQueue: %s Error on LPop: %v", r.name, err)
time.Sleep(time.Millisecond * 100)
continue
}
if len(bs) == 0 {
time.Sleep(time.Millisecond * 100)
continue
}
var data Data
if r.exemplar != nil {
t := reflect.TypeOf(r.exemplar)
n := reflect.New(t)
ne := n.Elem()
err = json.Unmarshal(bs, ne.Addr().Interface())
data = ne.Interface().(Data)
} else {
err = json.Unmarshal(bs, &data)
}
if err != nil {
log.Error("RedisQueue: %s Error on Unmarshal: %v", r.name, err)
time.Sleep(time.Millisecond * 100)
continue
}
log.Trace("RedisQueue: %s Task found: %#v", r.name, data)
r.pool.Push(data)
}
}
}
// Push implements Queue
func (r *RedisQueue) Push(data Data) error {
if r.exemplar != nil {
// Assert data is of same type as r.exemplar
value := reflect.ValueOf(data)
t := value.Type()
exemplarType := reflect.ValueOf(r.exemplar).Type()
if !t.AssignableTo(exemplarType) || data == nil {
return fmt.Errorf("Unable to assign data: %v to same type as exemplar: %v in %s", data, r.exemplar, r.name)
}
}
bs, err := json.Marshal(data)
if err != nil {
return err
}
return r.client.RPush(r.queueName, bs).Err()
}
// Shutdown processing from this queue
func (r *RedisQueue) Shutdown() {
log.Trace("Shutdown: %s", r.name)
r.lock.Lock()
select {
case <-r.closed:
default:
close(r.closed)
}
r.lock.Unlock()
}
// Terminate this queue and close the queue
func (r *RedisQueue) Terminate() {
log.Trace("Terminating: %s", r.name)
r.Shutdown()
r.lock.Lock()
select {
case <-r.terminated:
r.lock.Unlock()
default:
close(r.terminated)
r.lock.Unlock()
if err := r.client.Close(); err != nil {
log.Error("Error whilst closing internal redis client in %s: %v", r.name, err)
}
}
}
// Name returns the name of this queue
func (r *RedisQueue) Name() string {
return r.name
}
func init() {
queuesMap[RedisQueueType] = NewRedisQueue
}

43
modules/queue/queue_test.go Normal file
View file

@ -0,0 +1,43 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
)
type testData struct {
TestString string
TestInt int
}
func TestToConfig(t *testing.T) {
cfg := testData{
TestString: "Config",
TestInt: 10,
}
exemplar := testData{}
cfg2I, err := toConfig(exemplar, cfg)
assert.NoError(t, err)
cfg2, ok := (cfg2I).(testData)
assert.True(t, ok)
assert.NotEqual(t, cfg2, exemplar)
assert.Equal(t, &cfg, &cfg2)
cfgString, err := json.Marshal(cfg)
assert.NoError(t, err)
cfg3I, err := toConfig(exemplar, cfgString)
assert.NoError(t, err)
cfg3, ok := (cfg3I).(testData)
assert.True(t, ok)
assert.Equal(t, cfg.TestString, cfg3.TestString)
assert.Equal(t, cfg.TestInt, cfg3.TestInt)
assert.NotEqual(t, cfg3, exemplar)
}

206
modules/queue/queue_wrapped.go Normal file
View file

@ -0,0 +1,206 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"fmt"
"reflect"
"sync"
"time"
"code.gitea.io/gitea/modules/log"
)
// WrappedQueueType is the type for a wrapped delayed starting queue
const WrappedQueueType Type = "wrapped"
// WrappedQueueConfiguration is the configuration for a WrappedQueue
type WrappedQueueConfiguration struct {
Underlying Type
Timeout time.Duration
MaxAttempts int
Config interface{}
QueueLength int
Name string
}
type delayedStarter struct {
lock sync.Mutex
internal Queue
underlying Type
cfg interface{}
timeout time.Duration
maxAttempts int
name string
}
// setInternal must be called with the lock locked.
func (q *delayedStarter) setInternal(atShutdown func(context.Context, func()), handle HandlerFunc, exemplar interface{}) error {
var ctx context.Context
var cancel context.CancelFunc
if q.timeout > 0 {
ctx, cancel = context.WithTimeout(context.Background(), q.timeout)
} else {
ctx, cancel = context.WithCancel(context.Background())
}
defer cancel()
// Ensure we also stop at shutdown
atShutdown(ctx, func() {
cancel()
})
i := 1
for q.internal == nil {
select {
case <-ctx.Done():
return fmt.Errorf("Timedout creating queue %v with cfg %v in %s", q.underlying, q.cfg, q.name)
default:
queue, err := NewQueue(q.underlying, handle, q.cfg, exemplar)
if err == nil {
q.internal = queue
q.lock.Unlock()
break
}
if err.Error() != "resource temporarily unavailable" {
log.Warn("[Attempt: %d] Failed to create queue: %v for %s cfg: %v error: %v", i, q.underlying, q.name, q.cfg, err)
}
i++
if q.maxAttempts > 0 && i > q.maxAttempts {
return fmt.Errorf("Unable to create queue %v for %s with cfg %v by max attempts: error: %v", q.underlying, q.name, q.cfg, err)
}
sleepTime := 100 * time.Millisecond
if q.timeout > 0 && q.maxAttempts > 0 {
sleepTime = (q.timeout - 200*time.Millisecond) / time.Duration(q.maxAttempts)
}
t := time.NewTimer(sleepTime)
select {
case <-ctx.Done():
t.Stop()
case <-t.C:
}
}
}
return nil
}
// WrappedQueue wraps a delayed starting queue
type WrappedQueue struct {
delayedStarter
handle HandlerFunc
exemplar interface{}
channel chan Data
}
// NewWrappedQueue will attempt to create a queue of the provided type,
// but if there is a problem creating this queue it will instead create
// a WrappedQueue with delayed startup of the queue instead and a
// channel which will be redirected to the queue
func NewWrappedQueue(handle HandlerFunc, cfg, exemplar interface{}) (Queue, error) {
configInterface, err := toConfig(WrappedQueueConfiguration{}, cfg)
if err != nil {
return nil, err
}
config := configInterface.(WrappedQueueConfiguration)
queue, err := NewQueue(config.Underlying, handle, config.Config, exemplar)
if err == nil {
// Just return the queue there is no need to wrap
return queue, nil
}
if IsErrInvalidConfiguration(err) {
// Retrying ain't gonna make this any better...
return nil, ErrInvalidConfiguration{cfg: cfg}
}
queue = &WrappedQueue{
handle: handle,
channel: make(chan Data, config.QueueLength),
exemplar: exemplar,
delayedStarter: delayedStarter{
cfg: config.Config,
underlying: config.Underlying,
timeout: config.Timeout,
maxAttempts: config.MaxAttempts,
name: config.Name,
},
}
_ = GetManager().Add(queue, WrappedQueueType, config, exemplar, nil)
return queue, nil
}
// Name returns the name of the queue
func (q *WrappedQueue) Name() string {
return q.name + "-wrapper"
}
// Push will push the data to the internal channel checking it against the exemplar
func (q *WrappedQueue) Push(data Data) error {
if q.exemplar != nil {
// Assert data is of same type as r.exemplar
value := reflect.ValueOf(data)
t := value.Type()
exemplarType := reflect.ValueOf(q.exemplar).Type()
if !t.AssignableTo(exemplarType) || data == nil {
return fmt.Errorf("Unable to assign data: %v to same type as exemplar: %v in %s", data, q.exemplar, q.name)
}
}
q.channel <- data
return nil
}
// Run starts to run the queue and attempts to create the internal queue
func (q *WrappedQueue) Run(atShutdown, atTerminate func(context.Context, func())) {
q.lock.Lock()
if q.internal == nil {
err := q.setInternal(atShutdown, q.handle, q.exemplar)
q.lock.Unlock()
if err != nil {
log.Fatal("Unable to set the internal queue for %s Error: %v", q.Name(), err)
return
}
go func() {
for data := range q.channel {
_ = q.internal.Push(data)
}
}()
} else {
q.lock.Unlock()
}
q.internal.Run(atShutdown, atTerminate)
log.Trace("WrappedQueue: %s Done", q.name)
}
// Shutdown this queue and stop processing
func (q *WrappedQueue) Shutdown() {
log.Trace("WrappedQueue: %s Shutdown", q.name)
q.lock.Lock()
defer q.lock.Unlock()
if q.internal == nil {
return
}
if shutdownable, ok := q.internal.(Shutdownable); ok {
shutdownable.Shutdown()
}
}
// Terminate this queue and close the queue
func (q *WrappedQueue) Terminate() {
log.Trace("WrappedQueue: %s Terminating", q.name)
q.lock.Lock()
defer q.lock.Unlock()
if q.internal == nil {
return
}
if shutdownable, ok := q.internal.(Shutdownable); ok {
shutdownable.Terminate()
}
}
func init() {
queuesMap[WrappedQueueType] = NewWrappedQueue
}

75
modules/queue/setting.go Normal file
View file

@ -0,0 +1,75 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"encoding/json"
"fmt"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"
)
func validType(t string) (Type, error) {
if len(t) == 0 {
return PersistableChannelQueueType, nil
}
for _, typ := range RegisteredTypes() {
if t == string(typ) {
return typ, nil
}
}
return PersistableChannelQueueType, fmt.Errorf("Unknown queue type: %s defaulting to %s", t, string(PersistableChannelQueueType))
}
// CreateQueue for name with provided handler and exemplar
func CreateQueue(name string, handle HandlerFunc, exemplar interface{}) Queue {
q := setting.GetQueueSettings(name)
opts := make(map[string]interface{})
opts["Name"] = name
opts["QueueLength"] = q.Length
opts["BatchLength"] = q.BatchLength
opts["DataDir"] = q.DataDir
opts["Addresses"] = q.Addresses
opts["Network"] = q.Network
opts["Password"] = q.Password
opts["DBIndex"] = q.DBIndex
opts["QueueName"] = q.QueueName
opts["Workers"] = q.Workers
opts["MaxWorkers"] = q.MaxWorkers
opts["BlockTimeout"] = q.BlockTimeout
opts["BoostTimeout"] = q.BoostTimeout
opts["BoostWorkers"] = q.BoostWorkers
typ, err := validType(q.Type)
if err != nil {
log.Error("Invalid type %s provided for queue named %s defaulting to %s", q.Type, name, string(typ))
}
cfg, err := json.Marshal(opts)
if err != nil {
log.Error("Unable to marshall generic options: %v Error: %v", opts, err)
log.Error("Unable to create queue for %s", name, err)
return nil
}
returnable, err := NewQueue(typ, handle, cfg, exemplar)
if q.WrapIfNecessary && err != nil {
log.Warn("Unable to create queue for %s: %v", name, err)
log.Warn("Attempting to create wrapped queue")
returnable, err = NewQueue(WrappedQueueType, handle, WrappedQueueConfiguration{
Underlying: Type(q.Type),
Timeout: q.Timeout,
MaxAttempts: q.MaxAttempts,
Config: cfg,
QueueLength: q.Length,
}, exemplar)
}
if err != nil {
log.Error("Unable to create queue for %s: %v", name, err)
return nil
}
return returnable
}

325
modules/queue/workerpool.go Normal file
View file

@ -0,0 +1,325 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package queue
import (
"context"
"sync"
"time"
"code.gitea.io/gitea/modules/log"
)
// WorkerPool takes
type WorkerPool struct {
lock sync.Mutex
baseCtx context.Context
cancel context.CancelFunc
cond *sync.Cond
qid int64
maxNumberOfWorkers int
numberOfWorkers int
batchLength int
handle HandlerFunc
dataChan chan Data
blockTimeout time.Duration
boostTimeout time.Duration
boostWorkers int
}
// Push pushes the data to the internal channel
func (p *WorkerPool) Push(data Data) {
p.lock.Lock()
if p.blockTimeout > 0 && p.boostTimeout > 0 && (p.numberOfWorkers <= p.maxNumberOfWorkers || p.maxNumberOfWorkers < 0) {
p.lock.Unlock()
p.pushBoost(data)
} else {
p.lock.Unlock()
p.dataChan <- data
}
}
func (p *WorkerPool) pushBoost(data Data) {
select {
case p.dataChan <- data:
default:
p.lock.Lock()
if p.blockTimeout <= 0 {
p.lock.Unlock()
p.dataChan <- data
return
}
ourTimeout := p.blockTimeout
timer := time.NewTimer(p.blockTimeout)
p.lock.Unlock()
select {
case p.dataChan <- data:
if timer.Stop() {
select {
case <-timer.C:
default:
}
}
case <-timer.C:
p.lock.Lock()
if p.blockTimeout > ourTimeout || (p.numberOfWorkers > p.maxNumberOfWorkers && p.maxNumberOfWorkers >= 0) {
p.lock.Unlock()
p.dataChan <- data
return
}
p.blockTimeout *= 2
ctx, cancel := context.WithCancel(p.baseCtx)
mq := GetManager().GetManagedQueue(p.qid)
boost := p.boostWorkers
if (boost+p.numberOfWorkers) > p.maxNumberOfWorkers && p.maxNumberOfWorkers >= 0 {
boost = p.maxNumberOfWorkers - p.numberOfWorkers
}
if mq != nil {
log.Warn("WorkerPool: %d (for %s) Channel blocked for %v - adding %d temporary workers for %s, block timeout now %v", p.qid, mq.Name, ourTimeout, boost, p.boostTimeout, p.blockTimeout)
start := time.Now()
pid := mq.RegisterWorkers(boost, start, false, start, cancel)
go func() {
<-ctx.Done()
mq.RemoveWorkers(pid)
cancel()
}()
} else {
log.Warn("WorkerPool: %d Channel blocked for %v - adding %d temporary workers for %s, block timeout now %v", p.qid, ourTimeout, p.boostWorkers, p.boostTimeout, p.blockTimeout)
}
go func() {
<-time.After(p.boostTimeout)
cancel()
p.lock.Lock()
p.blockTimeout /= 2
p.lock.Unlock()
}()
p.addWorkers(ctx, boost)
p.lock.Unlock()
p.dataChan <- data
}
}
}
// NumberOfWorkers returns the number of current workers in the pool
func (p *WorkerPool) NumberOfWorkers() int {
p.lock.Lock()
defer p.lock.Unlock()
return p.numberOfWorkers
}
// MaxNumberOfWorkers returns the maximum number of workers automatically added to the pool
func (p *WorkerPool) MaxNumberOfWorkers() int {
p.lock.Lock()
defer p.lock.Unlock()
return p.maxNumberOfWorkers
}
// BoostWorkers returns the number of workers for a boost
func (p *WorkerPool) BoostWorkers() int {
p.lock.Lock()
defer p.lock.Unlock()
return p.boostWorkers
}
// BoostTimeout returns the timeout of the next boost
func (p *WorkerPool) BoostTimeout() time.Duration {
p.lock.Lock()
defer p.lock.Unlock()
return p.boostTimeout
}
// BlockTimeout returns the timeout til the next boost
func (p *WorkerPool) BlockTimeout() time.Duration {
p.lock.Lock()
defer p.lock.Unlock()
return p.blockTimeout
}
// SetSettings sets the setable boost values
func (p *WorkerPool) SetSettings(maxNumberOfWorkers, boostWorkers int, timeout time.Duration) {
p.lock.Lock()
defer p.lock.Unlock()
p.maxNumberOfWorkers = maxNumberOfWorkers
p.boostWorkers = boostWorkers
p.boostTimeout = timeout
}
// SetMaxNumberOfWorkers sets the maximum number of workers automatically added to the pool
// Changing this number will not change the number of current workers but will change the limit
// for future additions
func (p *WorkerPool) SetMaxNumberOfWorkers(newMax int) {
p.lock.Lock()
defer p.lock.Unlock()
p.maxNumberOfWorkers = newMax
}
// AddWorkers adds workers to the pool - this allows the number of workers to go above the limit
func (p *WorkerPool) AddWorkers(number int, timeout time.Duration) context.CancelFunc {
var ctx context.Context
var cancel context.CancelFunc
start := time.Now()
end := start
hasTimeout := false
if timeout > 0 {
ctx, cancel = context.WithTimeout(p.baseCtx, timeout)
end = start.Add(timeout)
hasTimeout = true
} else {
ctx, cancel = context.WithCancel(p.baseCtx)
}
mq := GetManager().GetManagedQueue(p.qid)
if mq != nil {
pid := mq.RegisterWorkers(number, start, hasTimeout, end, cancel)
go func() {
<-ctx.Done()
mq.RemoveWorkers(pid)
cancel()
}()
log.Trace("WorkerPool: %d (for %s) adding %d workers with group id: %d", p.qid, mq.Name, number, pid)
} else {
log.Trace("WorkerPool: %d adding %d workers (no group id)", p.qid, number)
}
p.addWorkers(ctx, number)
return cancel
}
// addWorkers adds workers to the pool
func (p *WorkerPool) addWorkers(ctx context.Context, number int) {
for i := 0; i < number; i++ {
p.lock.Lock()
if p.cond == nil {
p.cond = sync.NewCond(&p.lock)
}
p.numberOfWorkers++
p.lock.Unlock()
go func() {
p.doWork(ctx)
p.lock.Lock()
p.numberOfWorkers--
if p.numberOfWorkers == 0 {
p.cond.Broadcast()
} else if p.numberOfWorkers < 0 {
// numberOfWorkers can't go negative but...
log.Warn("Number of Workers < 0 for QID %d - this shouldn't happen", p.qid)
p.numberOfWorkers = 0
p.cond.Broadcast()
}
p.lock.Unlock()
}()
}
}
// Wait for WorkerPool to finish
func (p *WorkerPool) Wait() {
p.lock.Lock()
defer p.lock.Unlock()
if p.cond == nil {
p.cond = sync.NewCond(&p.lock)
}
if p.numberOfWorkers <= 0 {
return
}
p.cond.Wait()
}
// CleanUp will drain the remaining contents of the channel
// This should be called after AddWorkers context is closed
func (p *WorkerPool) CleanUp(ctx context.Context) {
log.Trace("WorkerPool: %d CleanUp", p.qid)
close(p.dataChan)
for data := range p.dataChan {
p.handle(data)
select {
case <-ctx.Done():
log.Warn("WorkerPool: %d Cleanup context closed before finishing clean-up", p.qid)
return
default:
}
}
log.Trace("WorkerPool: %d CleanUp Done", p.qid)
}
func (p *WorkerPool) doWork(ctx context.Context) {
delay := time.Millisecond * 300
var data = make([]Data, 0, p.batchLength)
for {
select {
case <-ctx.Done():
if len(data) > 0 {
log.Trace("Handling: %d data, %v", len(data), data)
p.handle(data...)
}
log.Trace("Worker shutting down")
return
case datum, ok := <-p.dataChan:
if !ok {
// the dataChan has been closed - we should finish up:
if len(data) > 0 {
log.Trace("Handling: %d data, %v", len(data), data)
p.handle(data...)
}
log.Trace("Worker shutting down")
return
}
data = append(data, datum)
if len(data) >= p.batchLength {
log.Trace("Handling: %d data, %v", len(data), data)
p.handle(data...)
data = make([]Data, 0, p.batchLength)
}
default:
timer := time.NewTimer(delay)
select {
case <-ctx.Done():
if timer.Stop() {
select {
case <-timer.C:
default:
}
}
if len(data) > 0 {
log.Trace("Handling: %d data, %v", len(data), data)
p.handle(data...)
}
log.Trace("Worker shutting down")
return
case datum, ok := <-p.dataChan:
if timer.Stop() {
select {
case <-timer.C:
default:
}
}
if !ok {
// the dataChan has been closed - we should finish up:
if len(data) > 0 {
log.Trace("Handling: %d data, %v", len(data), data)
p.handle(data...)
}
log.Trace("Worker shutting down")
return
}
data = append(data, datum)
if len(data) >= p.batchLength {
log.Trace("Handling: %d data, %v", len(data), data)
p.handle(data...)
data = make([]Data, 0, p.batchLength)
}
case <-timer.C:
delay = time.Millisecond * 100
if len(data) > 0 {
log.Trace("Handling: %d data, %v", len(data), data)
p.handle(data...)
data = make([]Data, 0, p.batchLength)
}
}
}
}
}

159
modules/setting/queue.go Normal file
View file

@ -0,0 +1,159 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package setting
import (
"fmt"
"path"
"strconv"
"strings"
"time"
"code.gitea.io/gitea/modules/log"
)
// QueueSettings represent the settings for a queue from the ini
type QueueSettings struct {
DataDir string
Length int
BatchLength int
ConnectionString string
Type string
Network string
Addresses string
Password string
QueueName string
DBIndex int
WrapIfNecessary bool
MaxAttempts int
Timeout time.Duration
Workers int
MaxWorkers int
BlockTimeout time.Duration
BoostTimeout time.Duration
BoostWorkers int
}
// Queue settings
var Queue = QueueSettings{}
// GetQueueSettings returns the queue settings for the appropriately named queue
func GetQueueSettings(name string) QueueSettings {
q := QueueSettings{}
sec := Cfg.Section("queue." + name)
// DataDir is not directly inheritable
q.DataDir = path.Join(Queue.DataDir, name)
// QueueName is not directly inheritable either
q.QueueName = name + Queue.QueueName
for _, key := range sec.Keys() {
switch key.Name() {
case "DATADIR":
q.DataDir = key.MustString(q.DataDir)
case "QUEUE_NAME":
q.QueueName = key.MustString(q.QueueName)
}
}
if !path.IsAbs(q.DataDir) {
q.DataDir = path.Join(AppDataPath, q.DataDir)
}
sec.Key("DATADIR").SetValue(q.DataDir)
// The rest are...
q.Length = sec.Key("LENGTH").MustInt(Queue.Length)
q.BatchLength = sec.Key("BATCH_LENGTH").MustInt(Queue.BatchLength)
q.ConnectionString = sec.Key("CONN_STR").MustString(Queue.ConnectionString)
q.Type = sec.Key("TYPE").MustString(Queue.Type)
q.WrapIfNecessary = sec.Key("WRAP_IF_NECESSARY").MustBool(Queue.WrapIfNecessary)
q.MaxAttempts = sec.Key("MAX_ATTEMPTS").MustInt(Queue.MaxAttempts)
q.Timeout = sec.Key("TIMEOUT").MustDuration(Queue.Timeout)
q.Workers = sec.Key("WORKERS").MustInt(Queue.Workers)
q.MaxWorkers = sec.Key("MAX_WORKERS").MustInt(Queue.MaxWorkers)
q.BlockTimeout = sec.Key("BLOCK_TIMEOUT").MustDuration(Queue.BlockTimeout)
q.BoostTimeout = sec.Key("BOOST_TIMEOUT").MustDuration(Queue.BoostTimeout)
q.BoostWorkers = sec.Key("BOOST_WORKERS").MustInt(Queue.BoostWorkers)
q.Network, q.Addresses, q.Password, q.DBIndex, _ = ParseQueueConnStr(q.ConnectionString)
return q
}
// NewQueueService sets up the default settings for Queues
// This is exported for tests to be able to use the queue
func NewQueueService() {
sec := Cfg.Section("queue")
Queue.DataDir = sec.Key("DATADIR").MustString("queues/")
if !path.IsAbs(Queue.DataDir) {
Queue.DataDir = path.Join(AppDataPath, Queue.DataDir)
}
Queue.Length = sec.Key("LENGTH").MustInt(20)
Queue.BatchLength = sec.Key("BATCH_LENGTH").MustInt(20)
Queue.ConnectionString = sec.Key("CONN_STR").MustString(path.Join(AppDataPath, ""))
Queue.Type = sec.Key("TYPE").MustString("")
Queue.Network, Queue.Addresses, Queue.Password, Queue.DBIndex, _ = ParseQueueConnStr(Queue.ConnectionString)
Queue.WrapIfNecessary = sec.Key("WRAP_IF_NECESSARY").MustBool(true)
Queue.MaxAttempts = sec.Key("MAX_ATTEMPTS").MustInt(10)
Queue.Timeout = sec.Key("TIMEOUT").MustDuration(GracefulHammerTime + 30*time.Second)
Queue.Workers = sec.Key("WORKERS").MustInt(1)
Queue.MaxWorkers = sec.Key("MAX_WORKERS").MustInt(10)
Queue.BlockTimeout = sec.Key("BLOCK_TIMEOUT").MustDuration(1 * time.Second)
Queue.BoostTimeout = sec.Key("BOOST_TIMEOUT").MustDuration(5 * time.Minute)
Queue.BoostWorkers = sec.Key("BOOST_WORKERS").MustInt(5)
Queue.QueueName = sec.Key("QUEUE_NAME").MustString("_queue")
// Now handle the old issue_indexer configuration
section := Cfg.Section("queue.issue_indexer")
issueIndexerSectionMap := map[string]string{}
for _, key := range section.Keys() {
issueIndexerSectionMap[key.Name()] = key.Value()
}
if _, ok := issueIndexerSectionMap["TYPE"]; !ok {
switch Indexer.IssueQueueType {
case LevelQueueType:
section.Key("TYPE").SetValue("level")
case ChannelQueueType:
section.Key("TYPE").SetValue("persistable-channel")
case RedisQueueType:
section.Key("TYPE").SetValue("redis")
default:
log.Fatal("Unsupported indexer queue type: %v",
Indexer.IssueQueueType)
}
}
if _, ok := issueIndexerSectionMap["LENGTH"]; !ok {
section.Key("LENGTH").SetValue(fmt.Sprintf("%d", Indexer.UpdateQueueLength))
}
if _, ok := issueIndexerSectionMap["BATCH_LENGTH"]; !ok {
section.Key("BATCH_LENGTH").SetValue(fmt.Sprintf("%d", Indexer.IssueQueueBatchNumber))
}
if _, ok := issueIndexerSectionMap["DATADIR"]; !ok {
section.Key("DATADIR").SetValue(Indexer.IssueQueueDir)
}
if _, ok := issueIndexerSectionMap["CONN_STR"]; !ok {
section.Key("CONN_STR").SetValue(Indexer.IssueQueueConnStr)
}
}
// ParseQueueConnStr parses a queue connection string
func ParseQueueConnStr(connStr string) (network, addrs, password string, dbIdx int, err error) {
fields := strings.Fields(connStr)
for _, f := range fields {
items := strings.SplitN(f, "=", 2)
if len(items) < 2 {
continue
}
switch strings.ToLower(items[0]) {
case "network":
network = items[1]
case "addrs":
addrs = items[1]
case "password":
password = items[1]
case "db":
dbIdx, err = strconv.Atoi(items[1])
if err != nil {
return
}
}
}
return
}

1
modules/setting/setting.go
View file

@ -1093,4 +1093,5 @@ func NewServices() {
newMigrationsService()
newIndexerService()
newTaskService()
NewQueueService()
}

27
modules/setting/task.go
View file

@ -4,22 +4,15 @@
package setting
var (
// Task settings
Task = struct {
QueueType string
QueueLength int
QueueConnStr string
}{
QueueType: ChannelQueueType,
QueueLength: 1000,
QueueConnStr: "addrs=127.0.0.1:6379 db=0",
}
)
func newTaskService() {
sec := Cfg.Section("task")
Task.QueueType = sec.Key("QUEUE_TYPE").MustString(ChannelQueueType)
Task.QueueLength = sec.Key("QUEUE_LENGTH").MustInt(1000)
Task.QueueConnStr = sec.Key("QUEUE_CONN_STR").MustString("addrs=127.0.0.1:6379 db=0")
taskSec := Cfg.Section("task")
queueTaskSec := Cfg.Section("queue.task")
switch taskSec.Key("QUEUE_TYPE").MustString(ChannelQueueType) {
case ChannelQueueType:
queueTaskSec.Key("TYPE").MustString("persistable-channel")
case RedisQueueType:
queueTaskSec.Key("TYPE").MustString("redis")
}
queueTaskSec.Key("LENGTH").MustInt(taskSec.Key("QUEUE_LENGTH").MustInt(1000))
queueTaskSec.Key("CONN_STR").MustString(taskSec.Key("QUEUE_CONN_STR").MustString("addrs=127.0.0.1:6379 db=0"))
}

14
modules/task/queue.go
View file

@ -1,14 +0,0 @@
// Copyright 2019 Gitea. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package task
import "code.gitea.io/gitea/models"
// Queue defines an interface to run task queue
type Queue interface {
Run() error
Push(*models.Task) error
Stop()
}

48
modules/task/queue_channel.go
View file

@ -1,48 +0,0 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package task
import (
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/log"
)
var (
_ Queue = &ChannelQueue{}
)
// ChannelQueue implements
type ChannelQueue struct {
queue chan *models.Task
}
// NewChannelQueue create a memory channel queue
func NewChannelQueue(queueLen int) *ChannelQueue {
return &ChannelQueue{
queue: make(chan *models.Task, queueLen),
}
}
// Run starts to run the queue
func (c *ChannelQueue) Run() error {
for task := range c.queue {
err := Run(task)
if err != nil {
log.Error("Run task failed: %s", err.Error())
}
}
return nil
}
// Push will push the task ID to queue
func (c *ChannelQueue) Push(task *models.Task) error {
c.queue <- task
return nil
}
// Stop stop the queue
func (c *ChannelQueue) Stop() {
close(c.queue)
}

130
modules/task/queue_redis.go
View file

@ -1,130 +0,0 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package task
import (
"encoding/json"
"errors"
"strconv"
"strings"
"time"
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/log"
"github.com/go-redis/redis"
)
var (
_ Queue = &RedisQueue{}
)
type redisClient interface {
RPush(key string, args ...interface{}) *redis.IntCmd
LPop(key string) *redis.StringCmd
Ping() *redis.StatusCmd
}
// RedisQueue redis queue
type RedisQueue struct {
client redisClient
queueName string
closeChan chan bool
}
func parseConnStr(connStr string) (addrs, password string, dbIdx int, err error) {
fields := strings.Fields(connStr)
for _, f := range fields {
items := strings.SplitN(f, "=", 2)
if len(items) < 2 {
continue
}
switch strings.ToLower(items[0]) {
case "addrs":
addrs = items[1]
case "password":
password = items[1]
case "db":
dbIdx, err = strconv.Atoi(items[1])
if err != nil {
return
}
}
}
return
}
// NewRedisQueue creates single redis or cluster redis queue
func NewRedisQueue(addrs string, password string, dbIdx int) (*RedisQueue, error) {
dbs := strings.Split(addrs, ",")
var queue = RedisQueue{
queueName: "task_queue",
closeChan: make(chan bool),
}
if len(dbs) == 0 {
return nil, errors.New("no redis host found")
} else if len(dbs) == 1 {
queue.client = redis.NewClient(&redis.Options{
Addr: strings.TrimSpace(dbs[0]), // use default Addr
Password: password, // no password set
DB: dbIdx, // use default DB
})
} else {
// cluster will ignore db
queue.client = redis.NewClusterClient(&redis.ClusterOptions{
Addrs: dbs,
Password: password,
})
}
if err := queue.client.Ping().Err(); err != nil {
return nil, err
}
return &queue, nil
}
// Run starts to run the queue
func (r *RedisQueue) Run() error {
for {
select {
case <-r.closeChan:
return nil
case <-time.After(time.Millisecond * 100):
}
bs, err := r.client.LPop(r.queueName).Bytes()
if err != nil {
if err != redis.Nil {
log.Error("LPop failed: %v", err)
}
time.Sleep(time.Millisecond * 100)
continue
}
var task models.Task
err = json.Unmarshal(bs, &task)
if err != nil {
log.Error("Unmarshal task failed: %s", err.Error())
} else {
err = Run(&task)
if err != nil {
log.Error("Run task failed: %s", err.Error())
}
}
}
}
// Push implements Queue
func (r *RedisQueue) Push(task *models.Task) error {
bs, err := json.Marshal(task)
if err != nil {
return err
}
return r.client.RPush(r.queueName, bs).Err()
}
// Stop stop the queue
func (r *RedisQueue) Stop() {
r.closeChan <- true
}

41
modules/task/task.go
View file

@ -8,14 +8,15 @@ import (
"fmt"
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/graceful"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/migrations/base"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/queue"
"code.gitea.io/gitea/modules/structs"
)
// taskQueue is a global queue of tasks
var taskQueue Queue
var taskQueue queue.Queue
// Run a task
func Run(t *models.Task) error {
@ -23,38 +24,32 @@ func Run(t *models.Task) error {
case structs.TaskTypeMigrateRepo:
return runMigrateTask(t)
default:
return fmt.Errorf("Unknow task type: %d", t.Type)
return fmt.Errorf("Unknown task type: %d", t.Type)
}
}
// Init will start the service to get all unfinished tasks and run them
func Init() error {
switch setting.Task.QueueType {
case setting.ChannelQueueType:
taskQueue = NewChannelQueue(setting.Task.QueueLength)
case setting.RedisQueueType:
var err error
addrs, pass, idx, err := parseConnStr(setting.Task.QueueConnStr)
if err != nil {
return err
}
taskQueue, err = NewRedisQueue(addrs, pass, idx)
if err != nil {
return err
}
default:
return fmt.Errorf("Unsupported task queue type: %v", setting.Task.QueueType)
taskQueue = queue.CreateQueue("task", handle, &models.Task{})
if taskQueue == nil {
return fmt.Errorf("Unable to create Task Queue")
}
go func() {
if err := taskQueue.Run(); err != nil {
log.Error("taskQueue.Run end failed: %v", err)
}
}()
go graceful.GetManager().RunWithShutdownFns(taskQueue.Run)
return nil
}
func handle(data ...queue.Data) {
for _, datum := range data {
task := datum.(*models.Task)
if err := Run(task); err != nil {
log.Error("Run task failed: %v", err)
}
}
}
// MigrateRepository add migration repository to task
func MigrateRepository(doer, u *models.User, opts base.MigrateOptions) error {
task, err := models.CreateMigrateTask(doer, u, opts)

2
options/locale/locale_bg-BG.ini
View file

@ -733,6 +733,8 @@ monitor.desc=Описание
monitor.start=Начален час
monitor.execute_time=Време за изпълнение
notices.system_notice_list=Системни известия
notices.actions=Действия
notices.select_all=Избери всички

2
options/locale/locale_cs-CZ.ini
View file

@ -1944,6 +1944,8 @@ monitor.desc=Popis
monitor.start=Čas zahájení
monitor.execute_time=Doba provádění
notices.system_notice_list=Systémová oznámení
notices.view_detail_header=Zobrazit detaily oznámení
notices.actions=Akce

6
options/locale/locale_de-DE.ini
View file

@ -10,6 +10,7 @@ link_account=Account verbinden
register=Registrieren
website=Webseite
version=Version
powered_by=Powered by %s
page=Seite
template=Template
language=Sprache
@ -1053,6 +1054,7 @@ pulls.is_checking=Die Konfliktprüfung läuft noch. Bitte aktualisiere die Seite
pulls.required_status_check_failed=Einige erforderliche Prüfungen waren nicht erfolgreich.
pulls.required_status_check_administrator=Als Administrator kannst du diesen Pull-Request weiterhin zusammenführen.
pulls.blocked_by_approvals=Dieser Pull-Request hat noch nicht genügend Zustimmungen. %d von %d Zustimmungen erteilt.
pulls.blocked_by_rejection=Dieser Pull-Request hat Änderungen, die von einem offiziellen Reviewer angefragt wurden.
pulls.can_auto_merge_desc=Dieser Pull-Request kann automatisch zusammengeführt werden.
pulls.cannot_auto_merge_desc=Dieser Pull-Request kann nicht automatisch zusammengeführt werden, da es Konflikte gibt.
pulls.cannot_auto_merge_helper=Bitte manuell zusammenführen, um die Konflikte zu lösen.
@ -1416,6 +1418,8 @@ settings.update_protect_branch_success=Branch-Schutz für den Branch „%s“ wu
settings.remove_protected_branch_success=Branch-Schutz für den Branch „%s“ wurde deaktiviert.
settings.protected_branch_deletion=Branch-Schutz deaktivieren
settings.protected_branch_deletion_desc=Wenn du den Branch-Schutz deaktivierst, können alle Nutzer mit Schreibrechten auf den Branch pushen. Fortfahren?
settings.block_rejected_reviews=Merge bei abgelehnten Reviews blockieren
settings.block_rejected_reviews_desc=Mergen ist nicht möglich, wenn Änderungen durch offizielle Reviewer angefragt werden, auch wenn es genügend Zustimmungen gibt.
settings.default_branch_desc=Wähle einen Standardbranch für Pull-Requests und Code-Commits:
settings.choose_branch=Wähle einen Branch …
settings.no_protected_branch=Es gibt keine geschützten Branches.
@ -2022,6 +2026,8 @@ monitor.process.cancel=Prozess abbrechen
monitor.process.cancel_desc=Abbrechen eines Prozesses kann Datenverlust verursachen
monitor.process.cancel_notices=Abbrechen: <strong>%s</strong>?
notices.system_notice_list=Systemmitteilungen
notices.view_detail_header=Meldungsdetails ansehen
notices.actions=Aktionen

50
options/locale/locale_en-US.ini
View file

@ -1410,7 +1410,7 @@ settings.protect_check_status_contexts_list = Status checks found in the last we
settings.protect_required_approvals = Required approvals:
settings.protect_required_approvals_desc = Allow only to merge pull request with enough positive reviews.
settings.protect_approvals_whitelist_enabled = Restrict approvals to whitelisted users or teams
settings.protect_approvals_whitelist_enabled_desc = Only reviews from whitelisted users or teams will count to the required approvals. Without approval whitelist, reviews from anyone with write access count to the required approvals.
settings.protect_approvals_whitelist_enabled_desc = Only reviews from whitelisted users or teams will count to the required approvals. Without approval whitelist, reviews from anyone with write access count to the required approvals.
settings.protect_approvals_whitelist_users = Whitelisted reviewers:
settings.protect_approvals_whitelist_teams = Whitelisted teams for reviews:
settings.add_protected_branch = Enable protection
@ -2028,6 +2028,54 @@ monitor.execute_time = Execution Time
monitor.process.cancel = Cancel process
monitor.process.cancel_desc = Cancelling a process may cause data loss
monitor.process.cancel_notices = Cancel: <strong>%s</strong>?
monitor.queues = Queues
monitor.queue = Queue: %s
monitor.queue.name = Name
monitor.queue.type = Type
monitor.queue.exemplar = Exemplar Type
monitor.queue.numberworkers = Number of Workers
monitor.queue.maxnumberworkers = Max Number of Workers
monitor.queue.review = Review Config
monitor.queue.review_add = Review/Add Workers
monitor.queue.configuration = Initial Configuration
monitor.queue.nopool.title = No Worker Pool
monitor.queue.nopool.desc = This queue wraps other queues and does not itself have a worker pool.
monitor.queue.wrapped.desc = A wrapped queue wraps a slow starting queue, buffering queued requests in a channel. It does not have a worker pool itself.
monitor.queue.persistable-channel.desc = A persistable-channel wraps two queues, a channel queue that has its own worker pool and a level queue for persisted requests from previous shutdowns. It does not have a worker pool itself.
monitor.queue.pool.timeout = Timeout
monitor.queue.pool.addworkers.title = Add Workers
monitor.queue.pool.addworkers.submit = Add Workers
monitor.queue.pool.addworkers.desc = Add Workers to this pool with or without a timeout. If you set a timeout these workers will be removed from the pool after the timeout has lapsed.
monitor.queue.pool.addworkers.numberworkers.placeholder = Number of Workers
monitor.queue.pool.addworkers.timeout.placeholder = Set to 0 for no timeout
monitor.queue.pool.addworkers.mustnumbergreaterzero = Number of Workers to add must be greater than zero
monitor.queue.pool.addworkers.musttimeoutduration = Timeout must be a golang duration eg. 5m or be 0
monitor.queue.settings.title = Pool Settings
monitor.queue.settings.desc = Pools dynamically grow with a boost in response to their worker queue blocking. These changes will not affect current worker groups.
monitor.queue.settings.timeout = Boost Timeout
monitor.queue.settings.timeout.placeholder = Currently %[1]v
monitor.queue.settings.timeout.error = Timeout must be a golang duration eg. 5m or be 0
monitor.queue.settings.numberworkers = Boost Number of Workers
monitor.queue.settings.numberworkers.placeholder = Currently %[1]d
monitor.queue.settings.numberworkers.error = Number of Workers to add must be greater than or equal to zero
monitor.queue.settings.maxnumberworkers = Max Number of workers
monitor.queue.settings.maxnumberworkers.placeholder = Currently %[1]d
monitor.queue.settings.maxnumberworkers.error = Max number of workers must be a number
monitor.queue.settings.submit = Update Settings
monitor.queue.settings.changed = Settings Updated
monitor.queue.settings.blocktimeout = Current Block Timeout
monitor.queue.settings.blocktimeout.value = %[1]v
monitor.queue.pool.none = This queue does not have a Pool
monitor.queue.pool.added = Worker Group Added
monitor.queue.pool.max_changed = Maximum number of workers changed
monitor.queue.pool.workers.title = Active Worker Groups
monitor.queue.pool.workers.none = No worker groups.
monitor.queue.pool.cancel = Shutdown Worker Group
monitor.queue.pool.cancelling = Worker Group shutting down
monitor.queue.pool.cancel_notices = Shutdown this group of %s workers?
monitor.queue.pool.cancel_desc = Leaving a queue without any worker groups may cause requests to block indefinitely.
notices.system_notice_list = System Notices
notices.view_detail_header = View Notice Details

78
options/locale/locale_es-ES.ini
View file

@ -66,6 +66,7 @@ forks=Forks
activities=Actividad
pull_requests=Pull Requests
issues=Incidencias
milestones=Hitos
cancel=Cancelar
add=Añadir
@ -269,6 +270,7 @@ authorize_application_description=Si concede el acceso, podrá acceder y escribi
authorize_title=¿Autorizar a "%s" a acceder a su cuenta?
authorization_failed=Autorización fallida
authorization_failed_desc=La autorización ha fallado porque hemos detectado una solicitud no válida. Por favor, póngase en contacto con el mantenedor de la aplicación que ha intentado autorizar.
sspi_auth_failed=Fallo en la autenticación SSPI
[mail]
activate_account=Por favor, active su cuenta
@ -302,6 +304,8 @@ CommitChoice=Hacer commit de la elección
TreeName=Ruta del archivo
Content=Contenido
SSPISeparatorReplacement=Separador
SSPIDefaultLanguage=Idioma predeterminado
require_error=` no puede estar vacío.`
alpha_dash_error=` solo debe contener caracteres alfanuméricos, guiones medios ('-') y guiones bajos ('_').`
@ -317,6 +321,7 @@ glob_pattern_error=` el patrón globo no es válido: %s.`
unknown_error=Error desconocido:
captcha_incorrect=El código CAPTCHA no es correcto.
password_not_match=Las contraseñas no coinciden.
lang_select_error=Seleccione un idioma de la lista.
username_been_taken=El nombre de usuario ya está en uso.
repo_name_been_taken=El nombre del repositorio ya está usado.
@ -328,6 +333,11 @@ team_no_units_error=Permitir el acceso a por lo menos una sección del repositor
email_been_used=La dirección de correo electrónico ya está usada.
openid_been_used=La dirección OpenID '%s' ya está usada.
username_password_incorrect=El nombre de usuario o la contraseña son incorrectos.
password_complexity=La contraseña no cumple los requisitos de complejidad:
password_lowercase_one=Al menos una letra minúscula
password_uppercase_one=Al menos una letra mayúscula
password_digit_one=Al menos un dígito
password_special_one=Al menos un carácter especial (puntuación, corchetes, comillas, etc.)
enterred_invalid_repo_name=El nombre de repositorio que ha entrado es incorrecto.
enterred_invalid_owner_name=El nuevo nombre de usuario no es válido.
enterred_invalid_password=La contraseña que ha introducido es incorrecta.
@ -581,6 +591,7 @@ email_notifications.submit=Establecer preferencias de correo electrónico
owner=Propietario
repo_name=Nombre del repositorio
repo_name_helper=Un buen nombre de repositorio está compuesto por palabras clave cortas, memorables y únicas.
repo_size=Tamaño del repositorio
template=Plantilla
template_select=Seleccionar una plantilla.
template_helper=Hacer del repositorio una plantilla
@ -626,7 +637,11 @@ reactions_more=y %d más
template.items=Elementos de plantilla
template.git_content=Contenido Git (rama predeterminada)
template.git_hooks=Git Hooks
template.git_hooks_tooltip=Actualmente no puede modificar o eliminar git hooks una vez añadidos. Seleccione esta opción sólo si confía en el repositorio de plantillas.
template.webhooks=Webhooks
template.topics=Temas
template.avatar=Avatar
template.one_item=Debe seleccionar al menos un elemento de plantilla
template.invalid=Debe seleccionar una plantilla de repositorio
@ -865,6 +880,10 @@ issues.closed_title=Cerrada
issues.num_comments=%d comentarios
issues.commented_at=`comentado <a href="#%s">%s</a>`
issues.delete_comment_confirm=¿Seguro que deseas eliminar este comentario?
issues.context.copy_link=Copiar enlace
issues.context.quote_reply=Citar respuesta
issues.context.edit=Editar
issues.context.delete=Eliminar
issues.no_content=Aún no existe contenido.
issues.close_issue=Cerrar
issues.close_comment_issue=Comentar y cerrar
@ -874,6 +893,12 @@ issues.create_comment=Comentar
issues.closed_at=`cerró <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.reopened_at=`reabrió <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.commit_ref_at=`mencionada esta incidencia en un commit <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.ref_issue_from=`<a href="%[3]s">referenció esta incidencia %[4]s</a> <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.ref_pull_from=`<a href="%[3]s">referenció este pull request %[4]s</a> <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.ref_closing_from=`<a href="%[3]s">referenció un pull request %[4]s que cerrará esta incidencia</a> <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.ref_reopening_from=`<a href="%[3]s">referenció un pull request %[4]s que reabrirá esta incidencia</a> <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.ref_closed_from=`<a href="%[3]s">cerró esta incidencia %[4]s</a> <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.ref_reopened_from=`<a href="%[3]s">reabrió esta incidencia %[4]s</a> <a id="%[1]s" href="#%[1]s">%[2]s</a>`
issues.ref_from=`de %[1]s`
issues.poster=Autor
issues.collaborator=Colaborador
@ -933,6 +958,7 @@ issues.add_time=Añadir tiempo gastado manualmente
issues.add_time_short=Añadir tiempo gastado
issues.add_time_cancel=Cancelar
issues.add_time_history=`añadió tiempo gastado %s`
issues.del_time_history=`eliminado el tiempo gastado %s`
issues.add_time_hours=Horas
issues.add_time_minutes=Minutos
issues.add_time_sum_to_small=No se ha entrado tiempo.
@ -1006,6 +1032,7 @@ pulls.no_results=Sin resultados.
pulls.nothing_to_compare=Estas ramas son iguales. No hay necesidad para crear un pull request.
pulls.has_pull_request=`Ya existe un pull request entre estas ramas: <a href="%[1]s/pulls/%[3]d">%[2]s#%[3]d</a>`
pulls.create=Crear Pull Request
pulls.title_desc=desea fusionar %[1]d commits de <code>%[2]s</code> en <code id="branch_target">%[3]s</code>
pulls.merged_title_desc=fusionados %[1]d commits de <code>%[2]s</code> en <code>%[3]s</code> %[4]s
pulls.tab_conversation=Conversación
pulls.tab_commits=Commits
@ -1014,6 +1041,7 @@ pulls.reopen_to_merge=Vuelva a abrir este Pull Request para realizar una fusión
pulls.cant_reopen_deleted_branch=Este pull request no se puede reabrir porque la rama fue eliminada.
pulls.merged=Fusionado
pulls.merged_as=El Pull Request se ha fusionado como <a rel="nofollow" class="ui sha" href="%[1]s"><code>%[2]s</code></a>.
pulls.is_closed=El pull request ha sido cerrado.
pulls.has_merged=El pull request ha sido fusionado.
pulls.title_wip_desc=`<a href="#">Comience el título con <strong>%s</strong></a> para prevenir que el pull request se fusione accidentalmente.`
pulls.cannot_merge_work_in_progress=Este pull request está marcado como un trabajo en progreso. Elimine el prefijo <strong>%s</strong> del título cuando esté listo
@ -1036,6 +1064,9 @@ pulls.rebase_merge_commit_pull_request=Hacer Rebase y Fusionar (--no-ff)
pulls.squash_merge_pull_request=Hacer Squash y Fusionar
pulls.invalid_merge_option=No puede utilizar esta opción de combinación para esta solicitud de extracción.
pulls.merge_conflict=Fusión fallida: Hubo un conflicto mientras se fusionaba: %[1]s<br>%[2]s<br>Pista: Pruebe una estrategia diferente
pulls.rebase_conflict=Fusión fallida: Hubo un conflicto mientras se rebasaba el commit: %[1]s<br>%[2]s<br>%[3]s<br>Sugerencia:Prueba una estrategia diferente
pulls.unrelated_histories=Fusionar Fallidos: El jefe de fusión y la base no comparten un historial común. Pista: Prueba una estrategia diferente
pulls.merge_out_of_date=Fusión fallida: Mientras se generaba la fusión, la base fue actualizada. Pista: Inténtelo de nuevo.
pulls.open_unmerged_pull_exists=`No puede realizar la reapertura porque hay un pull request pendiente (#%d) con propiedades idénticas.`
pulls.status_checking=Algunas comprobaciones están pendientes
pulls.status_checks_success=Todas las comprobaciones han sido exitosas
@ -1352,6 +1383,10 @@ settings.protected_branch_can_push_yes=Puede hacer push
settings.protected_branch_can_push_no=No puede hacer push
settings.branch_protection=Proteccion de la rama '<b>%s</b>'
settings.protect_this_branch=Activar protección de rama
settings.protect_disable_push=Deshabilitar Push
settings.protect_disable_push_desc=No se permitirá hacer push a esta rama.
settings.protect_enable_push=Habilitar Push
settings.protect_enable_push_desc=Cualquier usuario con permiso de escritura podrá hacer push a esta rama (pero no push --force).
settings.protect_whitelist_deploy_keys=Poner en lista blanca las claves de implementación con permisos de hacer push
settings.protect_whitelist_users=Usuarios en la lista blanca para hacer push:
settings.protect_whitelist_search_users=Buscar usuarios…
@ -1365,6 +1400,7 @@ settings.protect_check_status_contexts=Habilitar comprobación de estado
settings.protect_check_status_contexts_desc=Requiere comprobaciones de estado para pasar antes de fusionar. Elija qué los controles de estado deben pasar antes de que las ramas puedan ser fusionadas en una rama que coincida con esta regla. Cuando está habilitada, los commits deben ser primero empujados a otra rama y luego fusionados, o empujados directamente a una rama que coincida con esta regla después de que hayan pasado los comprobaciones de estado. Si no se selecciona ningún contexto, la última confirmación debe tener éxito independientemente del contexto.
settings.protect_check_status_contexts_list=Comprobaciones de estado para este repositorio encontradas durante la semana pasada
settings.protect_required_approvals=Aprobaciones requeridas:
settings.protect_required_approvals_desc=Permite fusionar sólo los pull request con suficientes comentarios positivos.
settings.protect_approvals_whitelist_users=Lista blanca de usuarios revisores:
settings.protect_approvals_whitelist_teams=Lista blanca de equipos revisores:
settings.add_protected_branch=Activar protección
@ -1398,7 +1434,20 @@ settings.lfs_filelist=No hay archivos LFS almacenados en este repositorio
settings.lfs_no_lfs_files=No hay archivos LFS almacenados en este repositorio
settings.lfs_findcommits=Encontrar consignas
settings.lfs_lfs_file_no_commits=No se encontraron commits para este archivo LFS
settings.lfs_noattribute=Esta ruta no tiene el atributo bloqueable en la rama por defecto
settings.lfs_delete=Eliminar archivo LFS con el OID %s
settings.lfs_delete_warning=Eliminar un archivo LFS puede causar errores de 'objeto no existe' en la compra. ¿Está seguro?
settings.lfs_findpointerfiles=Buscar de punteros LFS
settings.lfs_locks=Bloqueos
settings.lfs_invalid_locking_path=Ruta no válida: %s
settings.lfs_invalid_lock_directory=No se puede bloquear el directorio: %s
settings.lfs_lock_already_exists=El bloqueo ya existe: %s
settings.lfs_lock=Bloquear
settings.lfs_lock_path=Ruta del archivo a bloquear...
settings.lfs_locks_no_locks=Sin bloqueos
settings.lfs_lock_file_no_exist=El archivo bloqueado no existe en la rama por defecto
settings.lfs_force_unlock=Forzar desbloqueo
settings.lfs_pointers.found=Encontrados %d punteros - %d asociados, %d no asociados (%d falta en el almacén)
settings.lfs_pointers.sha=Blob SHA
settings.lfs_pointers.oid=OID
settings.lfs_pointers.inRepo=En repositorio
@ -1522,6 +1571,7 @@ team_name=Nombre del equipo
team_desc=Descripción
team_name_helper=Los nombres de equipos deben ser cortos y destacados.
team_desc_helper=Describa el propósito o rol del equipo.
team_access_desc=Acceso al repositorio
team_permission_desc=Permiso
team_unit_desc=Permitir acceso a las secciones del repositorio
@ -1568,6 +1618,8 @@ members.invite_now=Invitar
teams.join=Unirse
teams.leave=Abandonar
teams.can_create_org_repo=Crear repositorios
teams.can_create_org_repo_helper=Los miembros pueden crear nuevos repositorios en la organización. El creador obtendrá acceso al administrador del nuevo repositorio.
teams.read_access=Acceso de Lectura
teams.read_access_helper=Los miembros pueden ver y clonar los repositorios del equipo.
teams.write_access=Acceso de Escritura
@ -1587,15 +1639,24 @@ teams.delete_team_success=El equipo ha sido eliminado.
teams.read_permission_desc=Este equipo tiene permisos de <strong>Lectura</strong>: los miembros pueden ver y clonar los repositorios del equipo.
teams.write_permission_desc=Este equipo tiene permisos de <strong>Escritura</strong>: los miembros pueden ver y hacer push a los repositorios del equipo.
teams.admin_permission_desc=Este equipo tiene permisos de <strong>Administración</strong>: los miembros pueden ver, hacer push y añadir colaboradores a los repositorios del equipo.
teams.create_repo_permission_desc=Adicionalmente, este equipo concede permiso <strong>Crear repositorio</strong>: los miembros pueden crear nuevos repositorios en la organización.
teams.repositories=Repositorios del equipo
teams.search_repo_placeholder=Buscar repositorio…
teams.remove_all_repos_title=Eliminar todos los repositorios del equipo
teams.remove_all_repos_desc=Esto eliminará todos los repositorios del equipo.
teams.add_all_repos_title=Añadir todos los repositorios
teams.add_all_repos_desc=Esto añadirá todos los repositorios de la organización al equipo.
teams.add_nonexistent_repo=El repositorio que estás intentando añadir no existe, por favor, créalo primero.
teams.add_duplicate_users=El usuario ya es miembro del equipo.
teams.repos.none=Este equipo no tiene repositorios accesibles.
teams.members.none=No hay miembros en este equipo.
teams.specific_repositories=Repositorios específicos
teams.specific_repositories_helper=Los miembros sólo tendrán acceso a repositorios explícitamente agregados al equipo. Seleccionar este <strong>no</strong> eliminará automáticamente los repositorios ya añadidos con <i>Todos los repositorios</i>.
teams.all_repositories=Todos los repositorios
teams.all_repositories_helper=El equipo tiene acceso a todos los repositorios. Seleccionar esto <strong>añadirá todos los repositorios</strong> existentes al equipo.
teams.all_repositories_read_permission_desc=Este equipo concede <strong>Leer</strong> a <strong>todos los repositorios</strong>: los miembros pueden ver y clonar repositorios.
teams.all_repositories_write_permission_desc=Este equipo concede <strong>Escribir</strong> a <strong>todos los repositorios</strong>: los miembros pueden leer y enviar a los repositorios.
teams.all_repositories_admin_permission_desc=Este equipo concede a <strong>Administrador</strong> acceso a <strong>todos los repositorios</strong>: los miembros pueden leer, enviar y agregar colaboradores a los repositorios.
[admin]
dashboard=Panel de control
@ -1770,6 +1831,16 @@ auths.oauth2_authURL=URL de Autorización
auths.oauth2_profileURL=URL del perfil
auths.oauth2_emailURL=URL de correo
auths.enable_auto_register=Hablilitar Auto-Registro
auths.sspi_auto_create_users=Crear usuarios automáticamente
auths.sspi_auto_create_users_helper=Permitir al método de autenticación SSPI crear automáticamente nuevas cuentas para los usuarios que se conectan por primera vez
auths.sspi_auto_activate_users=Activar los usuarios automáticamente
auths.sspi_auto_activate_users_helper=Permitir al método de autenticación SSPI activar automáticamente los nuevos usuarios
auths.sspi_strip_domain_names=Eliminar los nombres de dominio de nombres de usuario
auths.sspi_strip_domain_names_helper=Si está marcado, los nombres de dominio se eliminarán de los nombres de inicio de sesión (por ejemplo, "DOMINIO\usuario" y "usuario@ejemplo.org" se convertirán en sólo "usuario").
auths.sspi_separator_replacement=Separador a usar en lugar de \, / y @
auths.sspi_separator_replacement_helper=El carácter a usar para reemplazar los separadores de los nombres de inicio de sesión de nivel inferior (por ejemplo, la \ en "DOMINIO\usuario") y en los nombres principales del usuario (por ejemplo, la @ en "user@example.org").
auths.sspi_default_language=Idioma predeterminado del usuario
auths.sspi_default_language_helper=Idioma predeterminado para los usuarios creados automáticamente por el método de autenticación SSPI. Deje vacío si prefiere que el idioma sea detectado automáticamente.
auths.tips=Consejos
auths.tips.oauth2.general=Autenticación OAuth2
auths.tips.oauth2.general.tip=Al registrar una nueva autenticación vía OAuth2, la URL de devolución/redirección debe ser: <host>/user/oauth2/<Nombre de Autenticación>/callback
@ -1795,6 +1866,7 @@ auths.delete_auth_desc=Eliminar un origen de autenticación impide que los usuar
auths.still_in_used=El orígen de autenticación todavía está en uso. Convierta o elimine cualquier usuario que utilice este origen de autenticación primero.
auths.deletion_success=El origen de autenticación ha sido eliminado.
auths.login_source_exist=El origen de autenticación '%s' ya existe.
auths.login_source_of_type_exist=Ya existe un origen de autenticación de este tipo.
config.server_config=Configuración del servidor
config.app_name=Título del sitio
@ -1939,6 +2011,11 @@ monitor.process=Procesos en ejecución
monitor.desc=Descripción
monitor.start=Hora de Inicio
monitor.execute_time=Tiempo de ejecución
monitor.process.cancel=Cancelar el proceso
monitor.process.cancel_desc=Cancelar un proceso puede ocasionar una pérdida de datos
monitor.process.cancel_notices=Cancelar: <strong>%s</strong>?
notices.system_notice_list=Notificaciones del Sistema
notices.view_detail_header=Ver detalles de notificación
@ -1965,6 +2042,7 @@ create_pull_request=`creado pull request <a href="%s/pulls/%s">%s#%[2]s</a>`
close_pull_request=`cerró el pull request <a href="%s/pulls/%s">%s#%[2]s</a>`
reopen_pull_request=`reabrió el pull request <a href="%s/pulls/%s">%s#%[2]s</a>`
comment_issue=`comentó en la incidencia <a href="%s/issues/%s">%s#%[2]s</a>`
comment_pull=`comentado en pull request <a href="%s/pulls/%s">%s#%[2]s</a>`
merge_pull_request=`fusionado pull request <a href="%s/pulls/%s">%s#%[2]s</a>`
transfer_repo=transfirió el repositorio <code>%s</code> a <a href="%s">%s</a>
push_tag=hizó push la etiqueta <a href="%s/src/tag/%s">%[2]s</a> a <a href="%[1]s">%[3]s</a>

2
options/locale/locale_fa-IR.ini
View file

@ -2026,6 +2026,8 @@ monitor.process.cancel=لغو فرآیند
monitor.process.cancel_desc=لغو کردن یک فرآیند ممکن است باعث از دست رفتن داده ها شود
monitor.process.cancel_notices=لغو: <strong>%s</strong>؟
notices.system_notice_list=هشدارهای سامانه
notices.view_detail_header=مشاهده جزئیات اخطار
notices.actions=اقدامات

2
options/locale/locale_fi-FI.ini
View file

@ -786,6 +786,8 @@ monitor.desc=Kuvaus
monitor.start=Alkamisaika
monitor.execute_time=Suoritusaika
notices.system_notice_list=Järjestelmän ilmoitukset
notices.actions=Toiminnot
notices.select_all=Valitse kaikki

4
options/locale/locale_fr-FR.ini
View file

@ -1194,7 +1194,7 @@ activity.git_stats_deletion_n=%d suppressions
search=Chercher
search.search_repo=Rechercher dans le dépôt
search.results=Résulats de la recherche « %s » dans <a href="%s"> %s</a>
search.results=Résultats de la recherche « %s » dans <a href="%s"> %s</a>
settings=Paramètres
settings.desc=Les paramètres sont l'endroit où gérer les options du dépôt
@ -2003,6 +2003,8 @@ monitor.process.cancel=Annuler le processus
monitor.process.cancel_desc=L'annulation d'un processus peut entraîner une perte de données
monitor.process.cancel_notices=Annuler : <strong>%s</strong>?
notices.system_notice_list=Informations
notices.view_detail_header=Voir les détails de l'information système
notices.actions=Actions

2
options/locale/locale_hu-HU.ini
View file

@ -911,6 +911,8 @@ monitor.desc=Leírás
monitor.start=Kezdés Időpontja
monitor.execute_time=Végrehajtási Idő
notices.system_notice_list=Rendszer Értesítések
notices.view_detail_header=Értesítés Részletei
notices.actions=Műveletek

2
options/locale/locale_id-ID.ini
View file

@ -957,6 +957,8 @@ monitor.desc=Deskripsi
monitor.start=Waktu mulai
monitor.execute_time=Waktu pelaksanaan
notices.system_notice_list=Pemberitahuan sistem
notices.view_detail_header=Lihat rincian pemberitahuan
notices.actions=Tindakan

2
options/locale/locale_it-IT.ini
View file

@ -1509,6 +1509,8 @@ monitor.desc=Descrizione
monitor.start=Orario Avvio
monitor.execute_time=Tempo di Esecuzione
notices.system_notice_list=Avvisi di Sistema
notices.view_detail_header=Visualizza dettagli dell'avviso
notices.actions=Azioni

6
options/locale/locale_ja-JP.ini
View file

@ -10,6 +10,7 @@ link_account=アカウント連携
register=登録
website=Webサイト
version=バージョン
powered_by=Powered by %s
page=ページ
template=テンプレート
language=言語
@ -1053,6 +1054,7 @@ pulls.is_checking=マージのコンフリクトを確認中です。 少し待
pulls.required_status_check_failed=いくつかの必要なステータスチェックが成功していません。
pulls.required_status_check_administrator=管理者であるため、このプルリクエストをマージすることは可能です。
pulls.blocked_by_approvals=このプルリクエストはまだ承認数が足りません。 %[1]d/%[2]dの承認を得ています。
pulls.blocked_by_rejection=このプルリクエストは公式レビューアにより変更要請されています。
pulls.can_auto_merge_desc=このプルリクエストは自動的にマージできます。
pulls.cannot_auto_merge_desc=コンフリクトが存在するため、このプルリクエストは自動的にマージできません。
pulls.cannot_auto_merge_helper=コンフリクトを解消するため手動でマージしてください。
@ -1416,6 +1418,8 @@ settings.update_protect_branch_success=ブランチ '%s' の保護を更新し
settings.remove_protected_branch_success=ブランチ '%s' の保護を無効にしました。
settings.protected_branch_deletion=ブランチ保護の無効化
settings.protected_branch_deletion_desc=ブランチ保護を無効にすると、書き込み権限を持つユーザーにブランチへのプッシュを許可することになります。 続行しますか？
settings.block_rejected_reviews=不承認レビューでマージをブロック
settings.block_rejected_reviews_desc=公式レビューアが変更を要請しているときは、承認数を満たしていても、マージできないようにします。
settings.default_branch_desc=プルリクエストやコミット表示のデフォルトのブランチを選択:
settings.choose_branch=ブランチを選択…
settings.no_protected_branch=保護しているブランチはありません。
@ -2022,6 +2026,8 @@ monitor.process.cancel=処理をキャンセル
monitor.process.cancel_desc=処理をキャンセルするとデータが失われる可能性があります
monitor.process.cancel_notices=キャンセル: <strong>%s</strong>?
notices.system_notice_list=システム通知
notices.view_detail_header=通知の詳細を表示
notices.actions=アクション

2
options/locale/locale_ko-KR.ini
View file

@ -1512,6 +1512,8 @@ monitor.desc=설명
monitor.start=시작 시간
monitor.execute_time=실행 시간
notices.system_notice_list=시스템 공지
notices.view_detail_header=알림 세부정보 보기
notices.actions=동작

2
options/locale/locale_lt-LT.ini
View file

@ -270,6 +270,8 @@ issues.save=Saugoti

2
options/locale/locale_lv-LV.ini
View file

@ -2026,6 +2026,8 @@ monitor.process.cancel=Atcelt procesu
monitor.process.cancel_desc=Procesa atcelšana var radīt datu zaudējumus
monitor.process.cancel_notices=Atcelt: <strong>%s</strong>?
notices.system_notice_list=Sistēmas paziņojumi
notices.view_detail_header=Skatīt paziņojuma detaļas
notices.actions=Darbības

2
options/locale/locale_ml-IN.ini
View file

@ -790,6 +790,8 @@ repos.issues=ഇഷ്യൂകള്‍
[action]
[tool]

2
options/locale/locale_nb-NO.ini
View file

@ -206,6 +206,8 @@ forgot_password=Glemt passord?

2
options/locale/locale_nl-NL.ini
View file

@ -1385,6 +1385,8 @@ monitor.desc=Omschrijving
monitor.start=Starttijd
monitor.execute_time=Uitvoertijd
notices.system_notice_list=Systeem aankondigingen
notices.actions=Acties
notices.select_all=Alles selecteren

2
options/locale/locale_nn-NO.ini
View file

@ -103,6 +103,8 @@

2
options/locale/locale_no-NO.ini
View file

@ -141,6 +141,8 @@ smtp_host=SMTP-vert

2
options/locale/locale_pl-PL.ini
View file

@ -2007,6 +2007,8 @@ monitor.process.cancel=Anuluj proces
monitor.process.cancel_desc=Anulowanie procesu może spowodować utratę danych
monitor.process.cancel_notices=Anuluj: <strong>%s</strong>?
notices.system_notice_list=Powiadomienia systemu
notices.view_detail_header=Pokaż szczegóły powiadomienia
notices.actions=Czynności

48
options/locale/locale_pt-BR.ini
View file

@ -2025,6 +2025,54 @@ monitor.execute_time=Tempo de execução
monitor.process.cancel=Cancelar processo
monitor.process.cancel_desc=Cancelar um processo pode causar perda de dados
monitor.process.cancel_notices=Cancelar: <strong>%s</strong>?
monitor.queues=Filas
monitor.queue=Fila: %s
monitor.queue.name=Nome
monitor.queue.type=Tipo
monitor.queue.exemplar=Tipo de modelo
monitor.queue.numberworkers=Número de executores
monitor.queue.maxnumberworkers=Número máximo de executores
monitor.queue.review=Revisar configuração
monitor.queue.review_add=Revisar/Adicionar executores
monitor.queue.configuration=Configuração inicial
monitor.queue.nopool.title=Nenhum conjunto de executores
monitor.queue.nopool.desc=Essa fila agrupa outras filas e não possui um conjunto de executores.
monitor.queue.wrapped.desc=Uma fila agrupada envolve uma fila inicial lenta, armazenando as solicitações da fila em um canal. Ela não possui um conjunto de executores em si.
monitor.queue.persistable-channel.desc=Um canal persistente envolve duas filas, uma fila de canais que tem seu próprio conjunto de executores e uma fila de nível para solicitações persistentes de encerramentos anteriores. Ela não tem um conjunto de executores em si.
monitor.queue.pool.timeout=Tempo de espera
monitor.queue.pool.addworkers.title=Adicionar executores
monitor.queue.pool.addworkers.submit=Adicionar executores
monitor.queue.pool.addworkers.desc=Adicionar executores a este conjunto com ou sem tempo de espera. Se você definir um tempo de espera, estes executores serão removidos do conjunto depois que o tempo de espera expirar.
monitor.queue.pool.addworkers.numberworkers.placeholder=Número de executores
monitor.queue.pool.addworkers.timeout.placeholder=Defina 0 para não ter tempo de espera
monitor.queue.pool.addworkers.mustnumbergreaterzero=O número de executores à adicionar deve ser maior que zero
monitor.queue.pool.addworkers.musttimeoutduration=Tempo de espera deve ser uma duração em "golang", por exemplo, 5m ou 0
monitor.queue.settings.title=Configurações do conjunto
monitor.queue.settings.desc=Os conjuntos crescem dinamicamente com um aumento em resposta ao bloqueio da fila de executores. Essas alterações não afetarão os grupos de executores atuais.
monitor.queue.settings.timeout=Tempo de espera do impulso
monitor.queue.settings.timeout.placeholder=Atualmente %[1]v
monitor.queue.settings.timeout.error=Tempo de espera deve ser uma duração em "golang", por exemplo, 5m ou 0
monitor.queue.settings.numberworkers=Número de executores a impulsionar
monitor.queue.settings.numberworkers.placeholder=Atualmente %[1]d
monitor.queue.settings.numberworkers.error=O número de executores à adicionar deve ser maior ou igual a zero
monitor.queue.settings.maxnumberworkers=Número máximo de executores
monitor.queue.settings.maxnumberworkers.placeholder=Atualmente %[1]d
monitor.queue.settings.maxnumberworkers.error=Número máximo de executores deve ser um número
monitor.queue.settings.submit=Atualizar configurações
monitor.queue.settings.changed=Configurações atualizadas
monitor.queue.settings.blocktimeout=Tempo de espera do bloqueio atual
monitor.queue.settings.blocktimeout.value=%[1]v
monitor.queue.pool.none=Esta fila não tem um conjunto
monitor.queue.pool.added=Grupo de executores adicionado
monitor.queue.pool.max_changed=Número máximo de executores alterado
monitor.queue.pool.workers.title=Grupo de executores ativo
monitor.queue.pool.workers.none=Nenhum grupo de executores.
monitor.queue.pool.cancel=Encerrar grupo de executores
monitor.queue.pool.cancelling=Encerrando grupo de executores
monitor.queue.pool.cancel_notices=Encerrar este grupo de %s executores?
monitor.queue.pool.cancel_desc=Deixar uma fila sem grupos de executores pode fazer com que as solicitações sejam bloqueadas indefinidamente.
notices.system_notice_list=Avisos do sistema
notices.view_detail_header=Ver detalhes do aviso

2
options/locale/locale_ru-RU.ini
View file

@ -1934,6 +1934,8 @@ monitor.process.cancel=Отменить процесс
monitor.process.cancel_desc=Отмена процесса может привести к потере данных
monitor.process.cancel_notices=Отменить: <strong>%s</strong>?
notices.system_notice_list=Уведомления системы
notices.view_detail_header=Подробности уведомления
notices.actions=Действия

2
options/locale/locale_sr-SP.ini
View file

@ -664,6 +664,8 @@ monitor.desc=Опис
monitor.start=Почетно време
monitor.execute_time=Време извршивања
notices.system_notice_list=Системска обавештавања
notices.actions=Акције
notices.select_all=Изабери све

2
options/locale/locale_sv-SE.ini
View file

@ -1545,6 +1545,8 @@ monitor.desc=Beskrivning
monitor.start=Starttid
monitor.execute_time=Exekveringstid
notices.system_notice_list=Systemnotiser
notices.view_detail_header=Visa notisdetaljer
notices.actions=Åtgärder

2
options/locale/locale_tr-TR.ini
View file

@ -2019,6 +2019,8 @@ monitor.process.cancel=İşlemi iptal et
monitor.process.cancel_desc=Bir işlemi iptal etmek veri kaybına neden olabilir
monitor.process.cancel_notices=İptal et: <strong>%s</strong>?
notices.system_notice_list=Sistem Bildirimleri
notices.view_detail_header=Bildirim Ayrıntılarını Görüntüle
notices.actions=İşlemler

2
options/locale/locale_uk-UA.ini
View file

@ -2021,6 +2021,8 @@ monitor.process.cancel=Зупинити процес
monitor.process.cancel_desc=Зупинка процесу може призвести до втрати даних
monitor.process.cancel_notices=Зупинити: <strong>%s</strong>?
notices.system_notice_list=Сповіщення системи
notices.view_detail_header=Переглянути деталі повідомлення
notices.actions=Дії

2
options/locale/locale_vi-VN.ini
View file

@ -103,6 +103,8 @@

48
options/locale/locale_zh-CN.ini
View file

@ -2025,6 +2025,54 @@ monitor.execute_time=执行时长
monitor.process.cancel=中止进程
monitor.process.cancel_desc=中止一个进程可能导致数据丢失
monitor.process.cancel_notices=中止：<strong>%s</strong> ？
monitor.queues=队列
monitor.queue=队列： %s
monitor.queue.name=名称
monitor.queue.type=类型
monitor.queue.exemplar=数据类型
monitor.queue.numberworkers=工作者数量
monitor.queue.maxnumberworkers=最大工作者数量
monitor.queue.review=查看配置
monitor.queue.review_add=查看/添加工作者
monitor.queue.configuration=初始配置
monitor.queue.nopool.title=没有工作者池
monitor.queue.nopool.desc=此队列包装其它队列，本身没有工作者池。
monitor.queue.wrapped.desc=一个包装队列包装一个启动缓慢队列，缓存队列请求到 channel 中。它本身没有一个工作者池。
monitor.queue.persistable-channel.desc=一个 persistable-channel 队列包装2个队列，一个 channel 队列拥有自己的工作者池，一个 level 队列用于永久存储。它没有自己的工作者池。
monitor.queue.pool.timeout=超时
monitor.queue.pool.addworkers.title=新增工作者
monitor.queue.pool.addworkers.submit=新增工作者
monitor.queue.pool.addworkers.desc=添加工作者到此池中。如果你设置了超时，这些工作者将会在超时结束后从池中移除。
monitor.queue.pool.addworkers.numberworkers.placeholder=工作者数量
monitor.queue.pool.addworkers.timeout.placeholder=设置为0则无超时
monitor.queue.pool.addworkers.mustnumbergreaterzero=要添加的工作者数量必须大于等于0
monitor.queue.pool.addworkers.musttimeoutduration=超时时间必须为Go语言时间间隔。例如 5m 或者 0
monitor.queue.settings.title=池设置
monitor.queue.settings.desc=池动态增长以应对队列阻塞。这些变更将不会影响当前的工作者组。
monitor.queue.settings.timeout=提高超时时间
monitor.queue.settings.timeout.placeholder=当前 %[1]v
monitor.queue.settings.timeout.error=超时时间必须为Go语言时间间隔。例如 5m 或者 0
monitor.queue.settings.numberworkers=提高工作者数量
monitor.queue.settings.numberworkers.placeholder=当前 %[1]d
monitor.queue.settings.numberworkers.error=要添加的工作者数量必须大于等于0
monitor.queue.settings.maxnumberworkers=最大工作者数量
monitor.queue.settings.maxnumberworkers.placeholder=当前 %[1]d
monitor.queue.settings.maxnumberworkers.error=最大工作者数必须是数字
monitor.queue.settings.submit=更新设置
monitor.queue.settings.changed=设置已更新
monitor.queue.settings.blocktimeout=当前阻塞超时时间
monitor.queue.settings.blocktimeout.value=%[1]v
monitor.queue.pool.none=此队列没有工作者池
monitor.queue.pool.added=工作者组添加成功
monitor.queue.pool.max_changed=最大工作者数量已更改
monitor.queue.pool.workers.title=活跃的工作者组
monitor.queue.pool.workers.none=没有工作者组。
monitor.queue.pool.cancel=停止工作者组
monitor.queue.pool.cancelling=工作者组正在关闭
monitor.queue.pool.cancel_notices=关掉这组 %s 工作者吗？
monitor.queue.pool.cancel_desc=没有工作者组的队列将会引起请求永久阻塞。
notices.system_notice_list=系统提示管理
notices.view_detail_header=查看提示详情

2
options/locale/locale_zh-HK.ini
View file

@ -795,6 +795,8 @@ monitor.desc=進程描述
monitor.start=開始時間
monitor.execute_time=已執行時間
notices.system_notice_list=系統提示管理
notices.view_detail_header=查看提示細節
notices.actions=操作

2
options/locale/locale_zh-TW.ini
View file

@ -1329,6 +1329,8 @@ monitor.desc=進程描述
monitor.start=開始時間
monitor.execute_time=已執行時間
notices.system_notice_list=系統提示管理
notices.view_detail_header=查看提示細節
notices.actions=操作

127
routers/admin/admin.go
View file

@ -11,6 +11,7 @@ import (
"net/url"
"os"
"runtime"
"strconv"
"strings"
"time"
@ -22,6 +23,7 @@ import (
"code.gitea.io/gitea/modules/graceful"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/process"
"code.gitea.io/gitea/modules/queue"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/timeutil"
"code.gitea.io/gitea/services/mailer"
@ -35,6 +37,7 @@ const (
tplDashboard base.TplName = "admin/dashboard"
tplConfig base.TplName = "admin/config"
tplMonitor base.TplName = "admin/monitor"
tplQueue base.TplName = "admin/queue"
)
var (
@ -355,6 +358,7 @@ func Monitor(ctx *context.Context) {
ctx.Data["PageIsAdminMonitor"] = true
ctx.Data["Processes"] = process.GetManager().Processes()
ctx.Data["Entries"] = cron.ListTasks()
ctx.Data["Queues"] = queue.GetManager().ManagedQueues()
ctx.HTML(200, tplMonitor)
}
@ -366,3 +370,126 @@ func MonitorCancel(ctx *context.Context) {
"redirect": ctx.Repo.RepoLink + "/admin/monitor",
})
}
// Queue shows details for a specific queue
func Queue(ctx *context.Context) {
qid := ctx.ParamsInt64("qid")
mq := queue.GetManager().GetManagedQueue(qid)
if mq == nil {
ctx.Status(404)
return
}
ctx.Data["Title"] = ctx.Tr("admin.monitor.queue", mq.Name)
ctx.Data["PageIsAdmin"] = true
ctx.Data["PageIsAdminMonitor"] = true
ctx.Data["Queue"] = mq
ctx.HTML(200, tplQueue)
}
// WorkerCancel cancels a worker group
func WorkerCancel(ctx *context.Context) {
qid := ctx.ParamsInt64("qid")
mq := queue.GetManager().GetManagedQueue(qid)
if mq == nil {
ctx.Status(404)
return
}
pid := ctx.ParamsInt64("pid")
mq.CancelWorkers(pid)
ctx.Flash.Info(ctx.Tr("admin.monitor.queue.pool.cancelling"))
ctx.JSON(200, map[string]interface{}{
"redirect": setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid),
})
}
// AddWorkers adds workers to a worker group
func AddWorkers(ctx *context.Context) {
qid := ctx.ParamsInt64("qid")
mq := queue.GetManager().GetManagedQueue(qid)
if mq == nil {
ctx.Status(404)
return
}
number := ctx.QueryInt("number")
if number < 1 {
ctx.Flash.Error(ctx.Tr("admin.monitor.queue.pool.addworkers.mustnumbergreaterzero"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
return
}
timeout, err := time.ParseDuration(ctx.Query("timeout"))
if err != nil {
ctx.Flash.Error(ctx.Tr("admin.monitor.queue.pool.addworkers.musttimeoutduration"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
return
}
if mq.Pool == nil {
ctx.Flash.Error(ctx.Tr("admin.monitor.queue.pool.none"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
return
}
mq.AddWorkers(number, timeout)
ctx.Flash.Success(ctx.Tr("admin.monitor.queue.pool.added"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
}
// SetQueueSettings sets the maximum number of workers and other settings for this queue
func SetQueueSettings(ctx *context.Context) {
qid := ctx.ParamsInt64("qid")
mq := queue.GetManager().GetManagedQueue(qid)
if mq == nil {
ctx.Status(404)
return
}
if mq.Pool == nil {
ctx.Flash.Error(ctx.Tr("admin.monitor.queue.pool.none"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
return
}
maxNumberStr := ctx.Query("max-number")
numberStr := ctx.Query("number")
timeoutStr := ctx.Query("timeout")
var err error
var maxNumber, number int
var timeout time.Duration
if len(maxNumberStr) > 0 {
maxNumber, err = strconv.Atoi(maxNumberStr)
if err != nil {
ctx.Flash.Error(ctx.Tr("admin.monitor.queue.settings.maxnumberworkers.error"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
return
}
if maxNumber < -1 {
maxNumber = -1
}
} else {
maxNumber = mq.MaxNumberOfWorkers()
}
if len(numberStr) > 0 {
number, err = strconv.Atoi(numberStr)
if err != nil || number < 0 {
ctx.Flash.Error(ctx.Tr("admin.monitor.queue.settings.numberworkers.error"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
return
}
} else {
number = mq.BoostWorkers()
}
if len(timeoutStr) > 0 {
timeout, err = time.ParseDuration(timeoutStr)
if err != nil {
ctx.Flash.Error(ctx.Tr("admin.monitor.queue.settings.timeout.error"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
return
}
} else {
timeout = mq.Pool.BoostTimeout()
}
mq.SetSettings(maxNumber, number, timeout)
ctx.Flash.Success(ctx.Tr("admin.monitor.queue.settings.changed"))
ctx.Redirect(setting.AppSubURL + fmt.Sprintf("/admin/monitor/queue/%d", qid))
}

7
routers/api/v1/api.go
View file

@ -661,9 +661,10 @@ func RegisterRoutes(m *macaron.Macaron) {
m.Group("/comments", func() {
m.Get("", repo.ListRepoIssueComments)
m.Group("/:id", func() {
m.Combo("", reqToken()).
Patch(mustNotBeArchived, bind(api.EditIssueCommentOption{}), repo.EditIssueComment).
Delete(repo.DeleteIssueComment)
m.Combo("").
Get(repo.GetIssueComment).
Patch(mustNotBeArchived, reqToken(), bind(api.EditIssueCommentOption{}), repo.EditIssueComment).
Delete(reqToken(), repo.DeleteIssueComment)
m.Combo("/reactions").
Get(repo.GetIssueCommentReactions).
Post(bind(api.EditReactionOption{}), reqToken(), repo.PostIssueCommentReaction).

81
routers/api/v1/repo/issue_comment.go
View file

@ -204,6 +204,74 @@ func CreateIssueComment(ctx *context.APIContext, form api.CreateIssueCommentOpti
ctx.JSON(http.StatusCreated, comment.APIFormat())
}
// GetIssueComment Get a comment by ID
func GetIssueComment(ctx *context.APIContext) {
// swagger:operation GET /repos/{owner}/{repo}/issues/comments/{id} issue issueGetComment
// ---
// summary: Get a comment
// consumes:
// - application/json
// produces:
// - application/json
// parameters:
// - name: owner
// in: path
// description: owner of the repo
// type: string
// required: true
// - name: repo
// in: path
// description: name of the repo
// type: string
// required: true
// - name: id
// in: path
// description: id of the comment
// type: integer
// format: int64
// required: true
// responses:
// "200":
// "$ref": "#/responses/Comment"
// "204":
// "$ref": "#/responses/empty"
// "403":
// "$ref": "#/responses/forbidden"
// "404":
// "$ref": "#/responses/notFound"
comment, err := models.GetCommentByID(ctx.ParamsInt64(":id"))
if err != nil {
if models.IsErrCommentNotExist(err) {
ctx.NotFound(err)
} else {
ctx.Error(http.StatusInternalServerError, "GetCommentByID", err)
}
return
}
if err = comment.LoadIssue(); err != nil {
ctx.InternalServerError(err)
return
}
if comment.Issue.RepoID != ctx.Repo.Repository.ID {
ctx.Status(http.StatusNotFound)
return
}
if comment.Type != models.CommentTypeComment {
ctx.Status(http.StatusNoContent)
return
}
if err := comment.LoadPoster(); err != nil {
ctx.Error(http.StatusInternalServerError, "comment.LoadPoster", err)
return
}
ctx.JSON(http.StatusOK, comment.APIFormat())
}
// EditIssueComment modify a comment of an issue
func EditIssueComment(ctx *context.APIContext, form api.EditIssueCommentOption) {
// swagger:operation PATCH /repos/{owner}/{repo}/issues/comments/{id} issue issueEditComment
@ -237,6 +305,13 @@ func EditIssueComment(ctx *context.APIContext, form api.EditIssueCommentOption)
// responses:
// "200":
// "$ref": "#/responses/Comment"
// "204":
// "$ref": "#/responses/empty"
// "403":
// "$ref": "#/responses/forbidden"
// "404":
// "$ref": "#/responses/notFound"
editIssueComment(ctx, form)
}
@ -283,6 +358,8 @@ func EditIssueCommentDeprecated(ctx *context.APIContext, form api.EditIssueComme
// "$ref": "#/responses/empty"
// "403":
// "$ref": "#/responses/forbidden"
// "404":
// "$ref": "#/responses/notFound"
editIssueComment(ctx, form)
}
@ -343,6 +420,8 @@ func DeleteIssueComment(ctx *context.APIContext) {
// "$ref": "#/responses/empty"
// "403":
// "$ref": "#/responses/forbidden"
// "404":
// "$ref": "#/responses/notFound"
deleteIssueComment(ctx)
}
@ -380,6 +459,8 @@ func DeleteIssueCommentDeprecated(ctx *context.APIContext) {
// "$ref": "#/responses/empty"
// "403":
// "$ref": "#/responses/forbidden"
// "404":
// "$ref": "#/responses/notFound"
deleteIssueComment(ctx)
}

3
routers/private/serv.go
View file

@ -15,6 +15,7 @@ import (
"code.gitea.io/gitea/modules/private"
"code.gitea.io/gitea/modules/setting"
repo_service "code.gitea.io/gitea/services/repository"
wiki_service "code.gitea.io/gitea/services/wiki"
"gitea.com/macaron/macaron"
)
@ -320,7 +321,7 @@ func ServCommand(ctx *macaron.Context) {
// Finally if we're trying to touch the wiki we should init it
if results.IsWiki {
if err = repo.InitWiki(); err != nil {
if err = wiki_service.InitWiki(repo); err != nil {
log.Error("Failed to initialize the wiki in %-v Error: %v", repo, err)
ctx.JSON(http.StatusInternalServerError, map[string]interface{}{
"results": results,

44
routers/repo/branch.go
View file

@ -16,6 +16,7 @@ import (
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/repofiles"
"code.gitea.io/gitea/modules/util"
"gopkg.in/src-d/go-git.v4/plumbing"
)
const (
@ -33,6 +34,7 @@ type Branch struct {
CommitsAhead int
CommitsBehind int
LatestPullRequest *models.PullRequest
MergeMovedOn bool
}
// Branches render repository branch page
@ -185,6 +187,12 @@ func loadBranches(ctx *context.Context) []*Branch {
return nil
}
repoIDToRepo := map[int64]*models.Repository{}
repoIDToRepo[ctx.Repo.Repository.ID] = ctx.Repo.Repository
repoIDToGitRepo := map[int64]*git.Repository{}
repoIDToGitRepo[ctx.Repo.Repository.ID] = ctx.Repo.GitRepo
branches := make([]*Branch, len(rawBranches))
for i := range rawBranches {
commit, err := rawBranches[i].GetCommit()
@ -213,11 +221,46 @@ func loadBranches(ctx *context.Context) []*Branch {
ctx.ServerError("GetLatestPullRequestByHeadInfo", err)
return nil
}
headCommit := commit.ID.String()
mergeMovedOn := false
if pr != nil {
pr.HeadRepo = ctx.Repo.Repository
if err := pr.LoadIssue(); err != nil {
ctx.ServerError("pr.LoadIssue", err)
return nil
}
if repo, ok := repoIDToRepo[pr.BaseRepoID]; ok {
pr.BaseRepo = repo
} else if err := pr.LoadBaseRepo(); err != nil {
ctx.ServerError("pr.LoadBaseRepo", err)
return nil
} else {
repoIDToRepo[pr.BaseRepoID] = pr.BaseRepo
}
if pr.HasMerged {
baseGitRepo, ok := repoIDToGitRepo[pr.BaseRepoID]
if !ok {
baseGitRepo, err = git.OpenRepository(pr.BaseRepo.RepoPath())
if err != nil {
ctx.ServerError("OpenRepository", err)
return nil
}
defer baseGitRepo.Close()
repoIDToGitRepo[pr.BaseRepoID] = baseGitRepo
}
pullCommit, err := baseGitRepo.GetRefCommitID(pr.GetGitRefName())
if err != nil && err != plumbing.ErrReferenceNotFound {
ctx.ServerError("GetBranchCommitID", err)
return nil
}
if err == nil && headCommit != pullCommit {
// the head has moved on from the merge - we shouldn't delete
mergeMovedOn = true
}
}
}
isIncluded := divergence.Ahead == 0 && ctx.Repo.Repository.DefaultBranch != branchName
@ -230,6 +273,7 @@ func loadBranches(ctx *context.Context) []*Branch {
CommitsAhead: divergence.Ahead,
CommitsBehind: divergence.Behind,
LatestPullRequest: pr,
MergeMovedOn: mergeMovedOn,
}
}

5
routers/repo/issue.go
View file

@ -966,7 +966,10 @@ func ViewIssue(ctx *context.Context) {
ctx.Data["IsBlockedByRejection"] = pull.ProtectedBranch.MergeBlockedByRejectedReview(pull)
ctx.Data["GrantedApprovals"] = cnt
}
ctx.Data["IsPullBranchDeletable"] = canDelete && pull.HeadRepo != nil && git.IsBranchExist(pull.HeadRepo.RepoPath(), pull.HeadBranch)
ctx.Data["IsPullBranchDeletable"] = canDelete &&
pull.HeadRepo != nil &&
git.IsBranchExist(pull.HeadRepo.RepoPath(), pull.HeadBranch) &&
(!pull.HasMerged || ctx.Data["HeadBranchCommitID"] == ctx.Data["PullHeadCommitID"])
ctx.Data["PullReviewers"], err = models.GetReviewersByIssueID(issue.ID)
if err != nil {

89
routers/repo/pull.go
View file

@ -330,25 +330,37 @@ func PrepareViewPullInfo(ctx *context.Context, issue *models.Issue) *git.Compare
repo := ctx.Repo.Repository
pull := issue.PullRequest
var err error
if err = pull.GetHeadRepo(); err != nil {
if err := pull.GetHeadRepo(); err != nil {
ctx.ServerError("GetHeadRepo", err)
return nil
}
if err := pull.GetBaseRepo(); err != nil {
ctx.ServerError("GetBaseRepo", err)
return nil
}
setMergeTarget(ctx, pull)
if err = pull.LoadProtectedBranch(); err != nil {
if err := pull.LoadProtectedBranch(); err != nil {
ctx.ServerError("GetLatestCommitStatus", err)
return nil
}
ctx.Data["EnableStatusCheck"] = pull.ProtectedBranch != nil && pull.ProtectedBranch.EnableStatusCheck
var headGitRepo *git.Repository
baseGitRepo, err := git.OpenRepository(pull.BaseRepo.RepoPath())
if err != nil {
ctx.ServerError("OpenRepository", err)
return nil
}
defer baseGitRepo.Close()
var headBranchExist bool
var headBranchSha string
// HeadRepo may be missing
if pull.HeadRepo != nil {
headGitRepo, err = git.OpenRepository(pull.HeadRepo.RepoPath())
var err error
headGitRepo, err := git.OpenRepository(pull.HeadRepo.RepoPath())
if err != nil {
ctx.ServerError("OpenRepository", err)
return nil
@ -358,46 +370,53 @@ func PrepareViewPullInfo(ctx *context.Context, issue *models.Issue) *git.Compare
headBranchExist = headGitRepo.IsBranchExist(pull.HeadBranch)
if headBranchExist {
sha, err := headGitRepo.GetBranchCommitID(pull.HeadBranch)
headBranchSha, err = headGitRepo.GetBranchCommitID(pull.HeadBranch)
if err != nil {
ctx.ServerError("GetBranchCommitID", err)
return nil
}
commitStatuses, err := models.GetLatestCommitStatus(repo, sha, 0)
if err != nil {
ctx.ServerError("GetLatestCommitStatus", err)
return nil
}
if len(commitStatuses) > 0 {
ctx.Data["LatestCommitStatuses"] = commitStatuses
ctx.Data["LatestCommitStatus"] = models.CalcCommitStatus(commitStatuses)
}
if pull.ProtectedBranch != nil && pull.ProtectedBranch.EnableStatusCheck {
ctx.Data["is_context_required"] = func(context string) bool {
for _, c := range pull.ProtectedBranch.StatusCheckContexts {
if c == context {
return true
}
}
return false
}
ctx.Data["IsRequiredStatusCheckSuccess"] = pull_service.IsCommitStatusContextSuccess(commitStatuses, pull.ProtectedBranch.StatusCheckContexts)
}
}
}
if pull.HeadRepo == nil || !headBranchExist {
ctx.Data["IsPullRequestBroken"] = true
ctx.Data["HeadTarget"] = "deleted"
ctx.Data["NumCommits"] = 0
ctx.Data["NumFiles"] = 0
sha, err := baseGitRepo.GetRefCommitID(pull.GetGitRefName())
if err != nil {
ctx.ServerError(fmt.Sprintf("GetRefCommitID(%s)", pull.GetGitRefName()), err)
return nil
}
compareInfo, err := headGitRepo.GetCompareInfo(models.RepoPath(repo.Owner.Name, repo.Name),
pull.BaseBranch, pull.HeadBranch)
commitStatuses, err := models.GetLatestCommitStatus(repo, sha, 0)
if err != nil {
ctx.ServerError("GetLatestCommitStatus", err)
return nil
}
if len(commitStatuses) > 0 {
ctx.Data["LatestCommitStatuses"] = commitStatuses
ctx.Data["LatestCommitStatus"] = models.CalcCommitStatus(commitStatuses)
}
if pull.ProtectedBranch != nil && pull.ProtectedBranch.EnableStatusCheck {
ctx.Data["is_context_required"] = func(context string) bool {
for _, c := range pull.ProtectedBranch.StatusCheckContexts {
if c == context {
return true
}
}
return false
}
ctx.Data["IsRequiredStatusCheckSuccess"] = pull_service.IsCommitStatusContextSuccess(commitStatuses, pull.ProtectedBranch.StatusCheckContexts)
}
ctx.Data["HeadBranchMovedOn"] = headBranchSha != sha
ctx.Data["HeadBranchCommitID"] = headBranchSha
ctx.Data["PullHeadCommitID"] = sha
if pull.HeadRepo == nil || !headBranchExist || headBranchSha != sha {
ctx.Data["IsPullRequestBroken"] = true
ctx.Data["HeadTarget"] = "deleted"
}
compareInfo, err := baseGitRepo.GetCompareInfo(pull.BaseRepo.RepoPath(),
pull.BaseBranch, pull.GetGitRefName())
if err != nil {
if strings.Contains(err.Error(), "fatal: Not a valid object name") {
ctx.Data["IsPullRequestBroken"] = true

43
routers/repo/wiki.go
View file

@ -22,6 +22,7 @@ import (
"code.gitea.io/gitea/modules/markup/markdown"
"code.gitea.io/gitea/modules/timeutil"
"code.gitea.io/gitea/modules/util"
wiki_service "code.gitea.io/gitea/services/wiki"
)
const (
@ -124,7 +125,7 @@ func wikiContentsByEntry(ctx *context.Context, entry *git.TreeEntry) []byte {
func wikiContentsByName(ctx *context.Context, commit *git.Commit, wikiName string) ([]byte, *git.TreeEntry, string, bool) {
var entry *git.TreeEntry
var err error
pageFilename := models.WikiNameToFilename(wikiName)
pageFilename := wiki_service.NameToFilename(wikiName)
if entry, err = findEntryForFile(commit, pageFilename); err != nil {
ctx.ServerError("findEntryForFile", err)
return nil, nil, "", false
@ -157,7 +158,7 @@ func renderViewPage(ctx *context.Context) (*git.Repository, *git.TreeEntry) {
if !entry.IsRegular() {
continue
}
wikiName, err := models.WikiFilenameToName(entry.Name())
wikiName, err := wiki_service.FilenameToName(entry.Name())
if err != nil {
if models.IsErrWikiInvalidFileName(err) {
continue
@ -172,17 +173,17 @@ func renderViewPage(ctx *context.Context) (*git.Repository, *git.TreeEntry) {
}
pages = append(pages, PageMeta{
Name: wikiName,
SubURL: models.WikiNameToSubURL(wikiName),
SubURL: wiki_service.NameToSubURL(wikiName),
})
}
ctx.Data["Pages"] = pages
// get requested pagename
pageName := models.NormalizeWikiName(ctx.Params(":page"))
pageName := wiki_service.NormalizeWikiName(ctx.Params(":page"))
if len(pageName) == 0 {
pageName = "Home"
}
ctx.Data["PageURL"] = models.WikiNameToSubURL(pageName)
ctx.Data["PageURL"] = wiki_service.NameToSubURL(pageName)
ctx.Data["old_title"] = pageName
ctx.Data["Title"] = pageName
ctx.Data["title"] = pageName
@ -243,11 +244,11 @@ func renderRevisionPage(ctx *context.Context) (*git.Repository, *git.TreeEntry)
}
// get requested pagename
pageName := models.NormalizeWikiName(ctx.Params(":page"))
pageName := wiki_service.NormalizeWikiName(ctx.Params(":page"))
if len(pageName) == 0 {
pageName = "Home"
}
ctx.Data["PageURL"] = models.WikiNameToSubURL(pageName)
ctx.Data["PageURL"] = wiki_service.NameToSubURL(pageName)
ctx.Data["old_title"] = pageName
ctx.Data["Title"] = pageName
ctx.Data["title"] = pageName
@ -320,11 +321,11 @@ func renderEditPage(ctx *context.Context) {
}()
// get requested pagename
pageName := models.NormalizeWikiName(ctx.Params(":page"))
pageName := wiki_service.NormalizeWikiName(ctx.Params(":page"))
if len(pageName) == 0 {
pageName = "Home"
}
ctx.Data["PageURL"] = models.WikiNameToSubURL(pageName)
ctx.Data["PageURL"] = wiki_service.NameToSubURL(pageName)
ctx.Data["old_title"] = pageName
ctx.Data["Title"] = pageName
ctx.Data["title"] = pageName
@ -474,7 +475,7 @@ func WikiPages(ctx *context.Context) {
ctx.ServerError("GetCommit", err)
return
}
wikiName, err := models.WikiFilenameToName(entry.Name())
wikiName, err := wiki_service.FilenameToName(entry.Name())
if err != nil {
if models.IsErrWikiInvalidFileName(err) {
continue
@ -488,7 +489,7 @@ func WikiPages(ctx *context.Context) {
}
pages = append(pages, PageMeta{
Name: wikiName,
SubURL: models.WikiNameToSubURL(wikiName),
SubURL: wiki_service.NameToSubURL(wikiName),
UpdatedUnix: timeutil.TimeStamp(c.Author.When.Unix()),
})
}
@ -528,7 +529,7 @@ func WikiRaw(ctx *context.Context) {
providedPath = providedPath[:len(providedPath)-3]
}
wikiPath := models.WikiNameToFilename(providedPath)
wikiPath := wiki_service.NameToFilename(providedPath)
entry, err = findEntryForFile(commit, wikiPath)
if err != nil {
ctx.ServerError("findFile", err)
@ -576,8 +577,8 @@ func NewWikiPost(ctx *context.Context, form auth.NewWikiForm) {
return
}
wikiName := models.NormalizeWikiName(form.Title)
if err := ctx.Repo.Repository.AddWikiPage(ctx.User, wikiName, form.Content, form.Message); err != nil {
wikiName := wiki_service.NormalizeWikiName(form.Title)
if err := wiki_service.AddWikiPage(ctx.User, ctx.Repo.Repository, wikiName, form.Content, form.Message); err != nil {
if models.IsErrWikiReservedName(err) {
ctx.Data["Err_Title"] = true
ctx.RenderWithErr(ctx.Tr("repo.wiki.reserved_page", wikiName), tplWikiNew, &form)
@ -590,7 +591,7 @@ func NewWikiPost(ctx *context.Context, form auth.NewWikiForm) {
return
}
ctx.Redirect(ctx.Repo.RepoLink + "/wiki/" + models.WikiNameToSubURL(wikiName))
ctx.Redirect(ctx.Repo.RepoLink + "/wiki/" + wiki_service.NameToSubURL(wikiName))
}
// EditWiki render wiki modify page
@ -623,25 +624,25 @@ func EditWikiPost(ctx *context.Context, form auth.NewWikiForm) {
return
}
oldWikiName := models.NormalizeWikiName(ctx.Params(":page"))
newWikiName := models.NormalizeWikiName(form.Title)
oldWikiName := wiki_service.NormalizeWikiName(ctx.Params(":page"))
newWikiName := wiki_service.NormalizeWikiName(form.Title)
if err := ctx.Repo.Repository.EditWikiPage(ctx.User, oldWikiName, newWikiName, form.Content, form.Message); err != nil {
if err := wiki_service.EditWikiPage(ctx.User, ctx.Repo.Repository, oldWikiName, newWikiName, form.Content, form.Message); err != nil {
ctx.ServerError("EditWikiPage", err)
return
}
ctx.Redirect(ctx.Repo.RepoLink + "/wiki/" + models.WikiNameToSubURL(newWikiName))
ctx.Redirect(ctx.Repo.RepoLink + "/wiki/" + wiki_service.NameToSubURL(newWikiName))
}
// DeleteWikiPagePost delete wiki page
func DeleteWikiPagePost(ctx *context.Context) {
wikiName := models.NormalizeWikiName(ctx.Params(":page"))
wikiName := wiki_service.NormalizeWikiName(ctx.Params(":page"))
if len(wikiName) == 0 {
wikiName = "Home"
}
if err := ctx.Repo.Repository.DeleteWikiPage(ctx.User, wikiName); err != nil {
if err := wiki_service.DeleteWikiPage(ctx.User, ctx.Repo.Repository, wikiName); err != nil {
ctx.ServerError("DeleteWikiPage", err)
return
}

3
routers/repo/wiki_test.go
View file

@ -13,6 +13,7 @@ import (
"code.gitea.io/gitea/modules/auth"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/test"
wiki_service "code.gitea.io/gitea/services/wiki"
"github.com/stretchr/testify/assert"
)
@ -29,7 +30,7 @@ func wikiEntry(t *testing.T, repo *models.Repository, wikiName string) *git.Tree
entries, err := commit.ListEntries()
assert.NoError(t, err)
for _, entry := range entries {
if entry.Name() == models.WikiNameToFilename(wikiName) {
if entry.Name() == wiki_service.NameToFilename(wikiName) {
return entry
}
}

12
routers/routes/routes.go
View file

@ -410,8 +410,16 @@ func RegisterRoutes(m *macaron.Macaron) {
m.Get("", adminReq, admin.Dashboard)
m.Get("/config", admin.Config)
m.Post("/config/test_mail", admin.SendTestMail)
m.Get("/monitor", admin.Monitor)
m.Post("/monitor/cancel/:pid", admin.MonitorCancel)
m.Group("/monitor", func() {
m.Get("", admin.Monitor)
m.Post("/cancel/:pid", admin.MonitorCancel)
m.Group("/queue/:qid", func() {
m.Get("", admin.Queue)
m.Post("/set", admin.SetQueueSettings)
m.Post("/add", admin.AddWorkers)
m.Post("/cancel/:pid", admin.WorkerCancel)
})
})
m.Group("/users", func() {
m.Get("", admin.Users)

1
services/pull/patch.go
View file

@ -171,7 +171,6 @@ func TestPatch(pr *models.PullRequest) error {
scanner := bufio.NewScanner(stderrReader)
for scanner.Scan() {
line := scanner.Text()
fmt.Printf("%s\n", line)
if strings.HasPrefix(line, prefix) {
conflict = true
filepath := strings.TrimSpace(strings.Split(line[len(prefix):], ":")[0])

322
services/wiki/wiki.go Normal file
View file

@ -0,0 +1,322 @@
// Copyright 2015 The Gogs Authors. All rights reserved.
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package wiki
import (
"fmt"
"net/url"
"os"
"strings"
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/sync"
"code.gitea.io/gitea/modules/util"
"github.com/unknwon/com"
)
var (
reservedWikiNames = []string{"_pages", "_new", "_edit", "raw"}
wikiWorkingPool = sync.NewExclusivePool()
)
func nameAllowed(name string) error {
if util.IsStringInSlice(name, reservedWikiNames) {
return models.ErrWikiReservedName{
Title: name,
}
}
return nil
}
// NameToSubURL converts a wiki name to its corresponding sub-URL.
func NameToSubURL(name string) string {
return url.QueryEscape(strings.Replace(name, " ", "-", -1))
}
// NormalizeWikiName normalizes a wiki name
func NormalizeWikiName(name string) string {
return strings.Replace(name, "-", " ", -1)
}
// NameToFilename converts a wiki name to its corresponding filename.
func NameToFilename(name string) string {
name = strings.Replace(name, " ", "-", -1)
return url.QueryEscape(name) + ".md"
}
// FilenameToName converts a wiki filename to its corresponding page name.
func FilenameToName(filename string) (string, error) {
if !strings.HasSuffix(filename, ".md") {
return "", models.ErrWikiInvalidFileName{
FileName: filename,
}
}
basename := filename[:len(filename)-3]
unescaped, err := url.QueryUnescape(basename)
if err != nil {
return "", err
}
return NormalizeWikiName(unescaped), nil
}
// InitWiki initializes a wiki for repository,
// it does nothing when repository already has wiki.
func InitWiki(repo *models.Repository) error {
if repo.HasWiki() {
return nil
}
if err := git.InitRepository(repo.WikiPath(), true); err != nil {
return fmt.Errorf("InitRepository: %v", err)
} else if err = models.CreateDelegateHooks(repo.WikiPath()); err != nil {
return fmt.Errorf("createDelegateHooks: %v", err)
}
return nil
}
// updateWikiPage adds a new page to the repository wiki.
func updateWikiPage(doer *models.User, repo *models.Repository, oldWikiName, newWikiName, content, message string, isNew bool) (err error) {
if err = nameAllowed(newWikiName); err != nil {
return err
}
wikiWorkingPool.CheckIn(com.ToStr(repo.ID))
defer wikiWorkingPool.CheckOut(com.ToStr(repo.ID))
if err = InitWiki(repo); err != nil {
return fmt.Errorf("InitWiki: %v", err)
}
hasMasterBranch := git.IsBranchExist(repo.WikiPath(), "master")
basePath, err := models.CreateTemporaryPath("update-wiki")
if err != nil {
return err
}
defer func() {
if err := models.RemoveTemporaryPath(basePath); err != nil {
log.Error("Merge: RemoveTemporaryPath: %s", err)
}
}()
cloneOpts := git.CloneRepoOptions{
Bare: true,
Shared: true,
}
if hasMasterBranch {
cloneOpts.Branch = "master"
}
if err := git.Clone(repo.WikiPath(), basePath, cloneOpts); err != nil {
log.Error("Failed to clone repository: %s (%v)", repo.FullName(), err)
return fmt.Errorf("Failed to clone repository: %s (%v)", repo.FullName(), err)
}
gitRepo, err := git.OpenRepository(basePath)
if err != nil {
log.Error("Unable to open temporary repository: %s (%v)", basePath, err)
return fmt.Errorf("Failed to open new temporary repository in: %s %v", basePath, err)
}
defer gitRepo.Close()
if hasMasterBranch {
if err := gitRepo.ReadTreeToIndex("HEAD"); err != nil {
log.Error("Unable to read HEAD tree to index in: %s %v", basePath, err)
return fmt.Errorf("Unable to read HEAD tree to index in: %s %v", basePath, err)
}
}
newWikiPath := NameToFilename(newWikiName)
if isNew {
filesInIndex, err := gitRepo.LsFiles(newWikiPath)
if err != nil {
log.Error("%v", err)
return err
}
if util.IsStringInSlice(newWikiPath, filesInIndex) {
return models.ErrWikiAlreadyExist{
Title: newWikiPath,
}
}
} else {
oldWikiPath := NameToFilename(oldWikiName)
filesInIndex, err := gitRepo.LsFiles(oldWikiPath)
if err != nil {
log.Error("%v", err)
return err
}
if util.IsStringInSlice(oldWikiPath, filesInIndex) {
err := gitRepo.RemoveFilesFromIndex(oldWikiPath)
if err != nil {
log.Error("%v", err)
return err
}
}
}
// FIXME: The wiki doesn't have lfs support at present - if this changes need to check attributes here
objectHash, err := gitRepo.HashObject(strings.NewReader(content))
if err != nil {
log.Error("%v", err)
return err
}
if err := gitRepo.AddObjectToIndex("100644", objectHash, newWikiPath); err != nil {
log.Error("%v", err)
return err
}
tree, err := gitRepo.WriteTree()
if err != nil {
log.Error("%v", err)
return err
}
commitTreeOpts := git.CommitTreeOpts{
Message: message,
}
sign, signingKey := repo.SignWikiCommit(doer)
if sign {
commitTreeOpts.KeyID = signingKey
} else {
commitTreeOpts.NoGPGSign = true
}
if hasMasterBranch {
commitTreeOpts.Parents = []string{"HEAD"}
}
commitHash, err := gitRepo.CommitTree(doer.NewGitSig(), tree, commitTreeOpts)
if err != nil {
log.Error("%v", err)
return err
}
if err := git.Push(basePath, git.PushOptions{
Remote: "origin",
Branch: fmt.Sprintf("%s:%s%s", commitHash.String(), git.BranchPrefix, "master"),
Env: models.FullPushingEnvironment(
doer,
doer,
repo,
repo.Name+".wiki",
0,
),
}); err != nil {
log.Error("%v", err)
return fmt.Errorf("Push: %v", err)
}
return nil
}
// AddWikiPage adds a new wiki page with a given wikiPath.
func AddWikiPage(doer *models.User, repo *models.Repository, wikiName, content, message string) error {
return updateWikiPage(doer, repo, "", wikiName, content, message, true)
}
// EditWikiPage updates a wiki page identified by its wikiPath,
// optionally also changing wikiPath.
func EditWikiPage(doer *models.User, repo *models.Repository, oldWikiName, newWikiName, content, message string) error {
return updateWikiPage(doer, repo, oldWikiName, newWikiName, content, message, false)
}
// DeleteWikiPage deletes a wiki page identified by its path.
func DeleteWikiPage(doer *models.User, repo *models.Repository, wikiName string) (err error) {
wikiWorkingPool.CheckIn(com.ToStr(repo.ID))
defer wikiWorkingPool.CheckOut(com.ToStr(repo.ID))
if err = InitWiki(repo); err != nil {
return fmt.Errorf("InitWiki: %v", err)
}
basePath, err := models.CreateTemporaryPath("update-wiki")
if err != nil {
return err
}
defer func() {
if err := models.RemoveTemporaryPath(basePath); err != nil {
log.Error("Merge: RemoveTemporaryPath: %s", err)
}
}()
if err := git.Clone(repo.WikiPath(), basePath, git.CloneRepoOptions{
Bare: true,
Shared: true,
Branch: "master",
}); err != nil {
log.Error("Failed to clone repository: %s (%v)", repo.FullName(), err)
return fmt.Errorf("Failed to clone repository: %s (%v)", repo.FullName(), err)
}
gitRepo, err := git.OpenRepository(basePath)
if err != nil {
log.Error("Unable to open temporary repository: %s (%v)", basePath, err)
return fmt.Errorf("Failed to open new temporary repository in: %s %v", basePath, err)
}
defer gitRepo.Close()
if err := gitRepo.ReadTreeToIndex("HEAD"); err != nil {
log.Error("Unable to read HEAD tree to index in: %s %v", basePath, err)
return fmt.Errorf("Unable to read HEAD tree to index in: %s %v", basePath, err)
}
wikiPath := NameToFilename(wikiName)
filesInIndex, err := gitRepo.LsFiles(wikiPath)
found := false
for _, file := range filesInIndex {
if file == wikiPath {
found = true
break
}
}
if found {
err := gitRepo.RemoveFilesFromIndex(wikiPath)
if err != nil {
return err
}
} else {
return os.ErrNotExist
}
// FIXME: The wiki doesn't have lfs support at present - if this changes need to check attributes here
tree, err := gitRepo.WriteTree()
if err != nil {
return err
}
message := "Delete page '" + wikiName + "'"
commitTreeOpts := git.CommitTreeOpts{
Message: message,
Parents: []string{"HEAD"},
}
sign, signingKey := repo.SignWikiCommit(doer)
if sign {
commitTreeOpts.KeyID = signingKey
} else {
commitTreeOpts.NoGPGSign = true
}
commitHash, err := gitRepo.CommitTree(doer.NewGitSig(), tree, commitTreeOpts)
if err != nil {
return err
}
if err := git.Push(basePath, git.PushOptions{
Remote: "origin",
Branch: fmt.Sprintf("%s:%s%s", commitHash.String(), git.BranchPrefix, "master"),
Env: models.PushingEnvironment(doer, repo),
}); err != nil {
return fmt.Errorf("Push: %v", err)
}
return nil
}

210
services/wiki/wiki_test.go Normal file
View file

@ -0,0 +1,210 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package wiki
import (
"path/filepath"
"testing"
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/git"
"github.com/stretchr/testify/assert"
)
func TestMain(m *testing.M) {
models.MainTest(m, filepath.Join("..", ".."))
}
func TestWikiNameToSubURL(t *testing.T) {
type test struct {
Expected string
WikiName string
}
for _, test := range []test{
{"wiki-name", "wiki name"},
{"wiki-name", "wiki-name"},
{"name-with%2Fslash", "name with/slash"},
{"name-with%25percent", "name with%percent"},
} {
assert.Equal(t, test.Expected, NameToSubURL(test.WikiName))
}
}
func TestNormalizeWikiName(t *testing.T) {
type test struct {
Expected string
WikiName string
}
for _, test := range []test{
{"wiki name", "wiki name"},
{"wiki name", "wiki-name"},
{"name with/slash", "name with/slash"},
{"name with%percent", "name-with%percent"},
{"%2F", "%2F"},
} {
assert.Equal(t, test.Expected, NormalizeWikiName(test.WikiName))
}
}
func TestWikiNameToFilename(t *testing.T) {
type test struct {
Expected string
WikiName string
}
for _, test := range []test{
{"wiki-name.md", "wiki name"},
{"wiki-name.md", "wiki-name"},
{"name-with%2Fslash.md", "name with/slash"},
{"name-with%25percent.md", "name with%percent"},
} {
assert.Equal(t, test.Expected, NameToFilename(test.WikiName))
}
}
func TestWikiFilenameToName(t *testing.T) {
type test struct {
Expected string
Filename string
}
for _, test := range []test{
{"hello world", "hello-world.md"},
{"symbols/?*", "symbols%2F%3F%2A.md"},
} {
name, err := FilenameToName(test.Filename)
assert.NoError(t, err)
assert.Equal(t, test.Expected, name)
}
for _, badFilename := range []string{
"nofileextension",
"wrongfileextension.txt",
} {
_, err := FilenameToName(badFilename)
assert.Error(t, err)
assert.True(t, models.IsErrWikiInvalidFileName(err))
}
_, err := FilenameToName("badescaping%%.md")
assert.Error(t, err)
assert.False(t, models.IsErrWikiInvalidFileName(err))
}
func TestWikiNameToFilenameToName(t *testing.T) {
// converting from wiki name to filename, then back to wiki name should
// return the original (normalized) name
for _, name := range []string{
"wiki-name",
"wiki name",
"wiki name with/slash",
"$$$%%%^^&&!@#$(),.<>",
} {
filename := NameToFilename(name)
resultName, err := FilenameToName(filename)
assert.NoError(t, err)
assert.Equal(t, NormalizeWikiName(name), resultName)
}
}
func TestRepository_InitWiki(t *testing.T) {
models.PrepareTestEnv(t)
// repo1 already has a wiki
repo1 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 1}).(*models.Repository)
assert.NoError(t, InitWiki(repo1))
// repo2 does not already have a wiki
repo2 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 2}).(*models.Repository)
assert.NoError(t, InitWiki(repo2))
assert.True(t, repo2.HasWiki())
}
func TestRepository_AddWikiPage(t *testing.T) {
assert.NoError(t, models.PrepareTestDatabase())
const wikiContent = "This is the wiki content"
const commitMsg = "Commit message"
repo := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 1}).(*models.Repository)
doer := models.AssertExistsAndLoadBean(t, &models.User{ID: 2}).(*models.User)
for _, wikiName := range []string{
"Another page",
"Here's a <tag> and a/slash",
} {
wikiName := wikiName
t.Run("test wiki exist: "+wikiName, func(t *testing.T) {
t.Parallel()
assert.NoError(t, AddWikiPage(doer, repo, wikiName, wikiContent, commitMsg))
// Now need to show that the page has been added:
gitRepo, err := git.OpenRepository(repo.WikiPath())
assert.NoError(t, err)
defer gitRepo.Close()
masterTree, err := gitRepo.GetTree("master")
assert.NoError(t, err)
wikiPath := NameToFilename(wikiName)
entry, err := masterTree.GetTreeEntryByPath(wikiPath)
assert.NoError(t, err)
assert.Equal(t, wikiPath, entry.Name(), "%s not addded correctly", wikiName)
})
}
t.Run("check wiki already exist", func(t *testing.T) {
t.Parallel()
// test for already-existing wiki name
err := AddWikiPage(doer, repo, "Home", wikiContent, commitMsg)
assert.Error(t, err)
assert.True(t, models.IsErrWikiAlreadyExist(err))
})
t.Run("check wiki reserved name", func(t *testing.T) {
t.Parallel()
// test for reserved wiki name
err := AddWikiPage(doer, repo, "_edit", wikiContent, commitMsg)
assert.Error(t, err)
assert.True(t, models.IsErrWikiReservedName(err))
})
}
func TestRepository_EditWikiPage(t *testing.T) {
const newWikiContent = "This is the new content"
const commitMsg = "Commit message"
repo := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 1}).(*models.Repository)
doer := models.AssertExistsAndLoadBean(t, &models.User{ID: 2}).(*models.User)
for _, newWikiName := range []string{
"Home", // same name as before
"New home",
"New/name/with/slashes",
} {
models.PrepareTestEnv(t)
assert.NoError(t, EditWikiPage(doer, repo, "Home", newWikiName, newWikiContent, commitMsg))
// Now need to show that the page has been added:
gitRepo, err := git.OpenRepository(repo.WikiPath())
assert.NoError(t, err)
masterTree, err := gitRepo.GetTree("master")
assert.NoError(t, err)
wikiPath := NameToFilename(newWikiName)
entry, err := masterTree.GetTreeEntryByPath(wikiPath)
assert.NoError(t, err)
assert.Equal(t, wikiPath, entry.Name(), "%s not editted correctly", newWikiName)
if newWikiName != "Home" {
_, err := masterTree.GetTreeEntryByPath("Home.md")
assert.Error(t, err)
}
gitRepo.Close()
}
}
func TestRepository_DeleteWikiPage(t *testing.T) {
models.PrepareTestEnv(t)
repo := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 1}).(*models.Repository)
doer := models.AssertExistsAndLoadBean(t, &models.User{ID: 2}).(*models.User)
assert.NoError(t, DeleteWikiPage(doer, repo, "Home"))
// Now need to show that the page has been added:
gitRepo, err := git.OpenRepository(repo.WikiPath())
assert.NoError(t, err)
defer gitRepo.Close()
masterTree, err := gitRepo.GetTree("master")
assert.NoError(t, err)
wikiPath := NameToFilename("Home")
_, err = masterTree.GetTreeEntryByPath(wikiPath)
assert.Error(t, err)
}

28
templates/admin/monitor.tmpl
View file

@ -31,6 +31,34 @@
</table>
</div>
<h4 class="ui top attached header">
{{.i18n.Tr "admin.monitor.queues"}}
</h4>
<div class="ui attached table segment">
<table class="ui very basic striped table">
<thead>
<tr>
<th>{{.i18n.Tr "admin.monitor.queue.name"}}</th>
<th>{{.i18n.Tr "admin.monitor.queue.type"}}</th>
<th>{{.i18n.Tr "admin.monitor.queue.exemplar"}}</th>
<th>{{.i18n.Tr "admin.monitor.queue.numberworkers"}}</th>
<th></th>
</tr>
</thead>
<tbody>
{{range .Queues}}
<tr>
<td>{{.Name}}</td>
<td>{{.Type}}</td>
<td>{{.ExemplarType}}</td>
<td>{{$sum := .NumberOfWorkers}}{{if lt $sum 0}}-{{else}}{{$sum}}{{end}}</td>
<td><a href="{{$.Link}}/queue/{{.QID}}" class="button">{{if lt $sum 0}}{{$.i18n.Tr "admin.monitor.queue.review"}}{{else}}{{$.i18n.Tr "admin.monitor.queue.review_add"}}{{end}}</a>
</tr>
{{end}}
</tbody>
</table>
</div>
<h4 class="ui top attached header">
{{.i18n.Tr "admin.monitor.process"}}
</h4>

147
templates/admin/queue.tmpl Normal file
View file

@ -0,0 +1,147 @@
{{template "base/head" .}}
<div class="admin monitor">
{{template "admin/navbar" .}}
<div class="ui container">
{{template "base/alert" .}}
<h4 class="ui top attached header">
{{.i18n.Tr "admin.monitor.queue" .Queue.Name}}
</h4>
<div class="ui attached table segment">
<table class="ui very basic striped table">
<thead>
<tr>
<th>{{.i18n.Tr "admin.monitor.queue.name"}}</th>
<th>{{.i18n.Tr "admin.monitor.queue.type"}}</th>
<th>{{.i18n.Tr "admin.monitor.queue.exemplar"}}</th>
<th>{{.i18n.Tr "admin.monitor.queue.numberworkers"}}</th>
<th>{{.i18n.Tr "admin.monitor.queue.maxnumberworkers"}}</th>
</tr>
</thead>
<tbody>
<tr>
<td>{{.Queue.Name}}</td>
<td>{{.Queue.Type}}</td>
<td>{{.Queue.ExemplarType}}</td>
<td>{{$sum := .Queue.NumberOfWorkers}}{{if lt $sum 0}}-{{else}}{{$sum}}{{end}}</td>
<td>{{if lt $sum 0}}-{{else}}{{.Queue.MaxNumberOfWorkers}}{{end}}</td>
</tr>
</tbody>
</table>
</div>
{{if lt $sum 0 }}
<h4 class="ui top attached header">
{{.i18n.Tr "admin.monitor.queue.nopool.title"}}
</h4>
<div class="ui attached segment">
{{if eq .Queue.Type "wrapped" }}
<p>{{.i18n.Tr "admin.monitor.queue.wrapped.desc"}}</p>
{{else if eq .Queue.Type "persistable-channel"}}
<p>{{.i18n.Tr "admin.monitor.queue.persistable-channel.desc"}}</p>
{{else}}
<p>{{.i18n.Tr "admin.monitor.queue.nopool.desc"}}</p>
{{end}}
</div>
{{else}}
<h4 class="ui top attached header">
{{.i18n.Tr "admin.monitor.queue.settings.title"}}
</h4>
<div class="ui attached segment">
<p>{{.i18n.Tr "admin.monitor.queue.settings.desc"}}</p>
<form method="POST" action="{{.Link}}/set">
{{$.CsrfTokenHtml}}
<div class="ui form">
<div class="inline field">
<label for="max-number">{{.i18n.Tr "admin.monitor.queue.settings.maxnumberworkers"}}</label>
<input name="max-number" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.settings.maxnumberworkers.placeholder" .Queue.MaxNumberOfWorkers}}">
</div>
<div class="inline field">
<label for="timeout">{{.i18n.Tr "admin.monitor.queue.settings.timeout"}}</label>
<input name="timeout" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.settings.timeout.placeholder" .Queue.BoostTimeout }}">
</div>
<div class="inline field">
<label for="number">{{.i18n.Tr "admin.monitor.queue.settings.numberworkers"}}</label>
<input name="number" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.settings.numberworkers.placeholder" .Queue.BoostWorkers}}">
</div>
<div class="inline field">
<label>{{.i18n.Tr "admin.monitor.queue.settings.blocktimeout"}}</label>
<span>{{.i18n.Tr "admin.monitor.queue.settings.blocktimeout.value" .Queue.BlockTimeout}}</span>
</div>
<button class="ui submit button">{{.i18n.Tr "admin.monitor.queue.settings.submit"}}</button>
</div>
</form>
</div>
<h4 class="ui top attached header">
{{.i18n.Tr "admin.monitor.queue.pool.addworkers.title"}}
</h4>
<div class="ui attached segment">
<p>{{.i18n.Tr "admin.monitor.queue.pool.addworkers.desc"}}</p>
<form method="POST" action="{{.Link}}/add">
{{$.CsrfTokenHtml}}
<div class="ui form">
<div class="fields">
<div class="field">
<label>{{.i18n.Tr "admin.monitor.queue.numberworkers"}}</label>
<input name="number" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.pool.addworkers.numberworkers.placeholder"}}">
</div>
<div class="field">
<label>{{.i18n.Tr "admin.monitor.queue.pool.timeout"}}</label>
<input name="timeout" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.pool.addworkers.timeout.placeholder"}}">
</div>
</div>
<button class="ui submit button">{{.i18n.Tr "admin.monitor.queue.pool.addworkers.submit"}}</button>
</div>
</form>
</div>
<h4 class="ui top attached header">
{{.i18n.Tr "admin.monitor.queue.pool.workers.title"}}
</h4>
<div class="ui attached table segment">
<table class="ui very basic striped table">
<thead>
<tr>
<th>{{.i18n.Tr "admin.monitor.queue.numberworkers"}}</th>
<th>{{.i18n.Tr "admin.monitor.start"}}</th>
<th>{{.i18n.Tr "admin.monitor.queue.pool.timeout"}}</th>
<th></th>
</tr>
</thead>
<tbody>
{{range .Queue.Workers}}
<tr>
<td>{{.Workers}}</td>
<td>{{DateFmtLong .Start}}</td>
<td>{{if .HasTimeout}}{{DateFmtLong .Timeout}}{{else}}-{{end}}</td>
<td>
<a class="delete-button" href="" data-url="{{$.Link}}/cancel/{{.PID}}" data-id="{{.PID}}" data-name="{{.Workers}}"><i class="close icon text red" title="{{$.i18n.Tr "remove"}}"></i></a>
</td>
</tr>
{{else}}
<tr>
<td colspan="4">{{.i18n.Tr "admin.monitor.queue.pool.workers.none" }}
</tr>
{{end}}
</tbody>
</table>
</div>
{{end}}
<h4 class="ui top attached header">
{{.i18n.Tr "admin.monitor.queue.configuration"}}
</h4>
<div class="ui attached segment">
<pre>{{.Queue.Configuration | JsonPrettyPrint}}
</div>
</div>
</div>
<div class="ui small basic delete modal">
<div class="ui icon header">
<i class="close icon"></i>
{{.i18n.Tr "admin.monitor.queue.pool.cancel"}}
</div>
<div class="content">
<p>{{$.i18n.Tr "admin.monitor.queue.pool.cancel_notices" `<span class="name"></span>` | Safe}}</p>
<p>{{$.i18n.Tr "admin.monitor.queue.pool.cancel_desc"}}</p>
</div>
{{template "base/delete_modal_actions" .}}
</div>
{{template "base/footer" .}}

6
templates/repo/branch/list.tmpl
View file

@ -84,6 +84,12 @@
<button id="new-pull-request" class="ui compact basic button">{{$.i18n.Tr "repo.pulls.compare_changes"}}</button>
</a>
{{end}}
{{else if and .LatestPullRequest.HasMerged .MergeMovedOn}}
{{if and (not .IsDeleted) $.AllowsPulls (gt .CommitsAhead 0)}}
<a href="{{$.RepoLink}}/compare/{{$.DefaultBranch | EscapePound}}...{{if ne $.Repository.Owner.Name $.Owner.Name}}{{$.Owner.Name}}:{{end}}{{.Name | EscapePound}}">
<button id="new-pull-request" class="ui compact basic button">{{$.i18n.Tr "repo.pulls.compare_changes"}}</button>
</a>
{{end}}
{{else}}
<a href="{{$.RepoLink}}/pulls/{{.LatestPullRequest.Issue.Index}}">#{{.LatestPullRequest.Issue.Index}}</a>
{{if .LatestPullRequest.HasMerged}}

4
templates/repo/issue/view_content/pull.tmpl
View file

@ -72,7 +72,7 @@
{{$.i18n.Tr "repo.pulls.reopen_to_merge"}}
{{end}}
</div>
{{if .IsPullBranchDeletable}}
{{if and .IsPullBranchDeletable ( not .IsPullRequestBroken )}}
<div class="ui divider"></div>
<div>
<a class="delete-button ui red button" href="" data-url="{{.DeleteBranchLink}}">{{$.i18n.Tr "repo.branch.delete" .HeadTarget}}</a>
@ -105,7 +105,7 @@
<div class="item text red">
<span class="octicon octicon-x"></span>
{{$.i18n.Tr "repo.pulls.blocked_by_rejection"}}
</div>
</div>
{{else if .Issue.PullRequest.IsChecking}}
<div class="item text yellow">
<span class="octicon octicon-sync"></span>

5
templates/repo/settings/collaboration.tmpl
View file

@ -79,9 +79,8 @@
{{end}}
</div>
{{if $allowedToChangeTeams}}
{{ $globalRepoAccess := (eq .LowerName "owners") }}
<div class="ui two wide column {{if $globalRepoAccess}}poping up{{end}}" {{if $globalRepoAccess}}data-content="This team has access to all repositories and can't be removed."{{end}}>
<button class="ui red tiny button inline text-thin delete-button {{if $globalRepoAccess}}disabled{{end}}" data-url="{{$.Link}}/team/delete" data-id="{{.ID}}">
<div class="ui two wide column {{if .IncludesAllRepositories}}poping up{{end}}" {{if .IncludesAllRepositories}}data-content="This team has access to all repositories and can't be removed."{{end}}>
<button class="ui red tiny button inline text-thin delete-button {{if .IncludesAllRepositories}}disabled{{end}}" data-url="{{$.Link}}/team/delete" data-id="{{.ID}}">
{{$.i18n.Tr "repo.settings.delete_collaborator"}}
</button>
</div>

69
templates/swagger/v1_json.tmpl
View file

@ -3002,6 +3002,57 @@
}
},
"/repos/{owner}/{repo}/issues/comments/{id}": {
"get": {
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"tags": [
"issue"
],
"summary": "Get a comment",
"operationId": "issueGetComment",
"parameters": [
{
"type": "string",
"description": "owner of the repo",
"name": "owner",
"in": "path",
"required": true
},
{
"type": "string",
"description": "name of the repo",
"name": "repo",
"in": "path",
"required": true
},
{
"type": "integer",
"format": "int64",
"description": "id of the comment",
"name": "id",
"in": "path",
"required": true
}
],
"responses": {
"200": {
"$ref": "#/responses/Comment"
},
"204": {
"$ref": "#/responses/empty"
},
"403": {
"$ref": "#/responses/forbidden"
},
"404": {
"$ref": "#/responses/notFound"
}
}
},
"delete": {
"tags": [
"issue"
@ -3038,6 +3089,9 @@
},
"403": {
"$ref": "#/responses/forbidden"
},
"404": {
"$ref": "#/responses/notFound"
}
}
},
@ -3087,6 +3141,15 @@
"responses": {
"200": {
"$ref": "#/responses/Comment"
},
"204": {
"$ref": "#/responses/empty"
},
"403": {
"$ref": "#/responses/forbidden"
},
"404": {
"$ref": "#/responses/notFound"
}
}
}
@ -3491,6 +3554,9 @@
},
"403": {
"$ref": "#/responses/forbidden"
},
"404": {
"$ref": "#/responses/notFound"
}
}
},
@ -3554,6 +3620,9 @@
},
"403": {
"$ref": "#/responses/forbidden"
},
"404": {
"$ref": "#/responses/notFound"
}
}
}