mirror of
https://github.com/discordeno/discordeno.git
synced 2026-09-17 08:47:22 +00:00
BREAKING: node migration + major improvements and style changes (#2783)
* Setup turborepo (#2610) * chore: BREAKING move to monorepo structure * chore: setup turborepo * setup more eslint and tsconfig (#2612) * fix packages setting lcoation * add dist to ignore * style * fix: .vscode set prettier and remove deno configs (#2611) * fix: rewrite to process.env * chore: add node types * style: run eslint --fix * fix: tests import ext * chore: fix formatter * chore: add build script * chore: remove site from turborepo * chore: move to seperate packages * chore: seperate util * chore: rename to index * fix: utils * chore: bump typescript * fix: in process of fixing rest * fix: logger * style * chore: fix turbo script * fix: types * fix: types * fix: shard * fix: gateway * fixing: bot * fix: at least it can run now * chore: fix turbo script * chore: move plugins * chore: add type and utils export * chore: working bot type and cache plugin * Add git hooks (#2618) * CI: Setup ci (#2669) * ci: fix test * ci: fix typo * fix: turbo script * fixes: yarn and linter errors in embeds pkg * chore: fix yarn gitignore * Node migration - devcontainer (#2672) * ✨ - feat: devcontainer -> node * 👷♂ - ci: devcontainer - add tabnine, prettier * fix: ignore .env and debug.ts * fix: couple of linter errors * fix: linter error * fix: gateway linter errors * style: update style * style: fix bot style * fix: type * ci: move all old workflow * chore: close #2619 * chore: close #2671 * test: add mocha * chore: add typescript plugin * test: add mocha * test: add test to utils * test/ci: update ci and coverage * chore: change script naming * ci: update include test * test: add coverage * ci: fix cache * ci: fix ci and codecov * Discordeno Documentation (#2673) * Add git hooks * Add documentation generatation * Change Documentation Engine * Add documentation * Remove autogenerated docs * combine lint staged action into one * style: fix Co-authored-by: H01001000 <heiheiho000@gmail.com> * chore: new package client * test: add rest test * ci: enable codecov * fix: type * test: add test to all packages * ci: add release to gh per commit * fix: ci syntax * fix: package version * fix: publish script * fix: remove private from gateway * ci: add filter for changes * fix: ci syntex * ci: try fix path filter * ci: try fix path filter * test: add test * ci: fix string and boolean * fix: package and ci * chore: fix turbo type cache * ci: also publish to npm * ci: change to public * fix: not publish to npm * fix: dependencies * chore: fix fmt script * fix: better rest typecheck Closes #2621 * fix: run yarn install * feat: add transformers to rest * feat: add helpers to rest * test: move bot utils test * reverse change to release.yml * chore: add clean build * refactor: discordeno * chore: add import type * chore: remove bot * fix: change deps from bot to dd * chore: update yarn lock * test: temp remove test from logger * refactor: remove transformers in helpers/channel * type: close #2622 * ci(fix): explicitly define coverage file * refactor: remove transformers in helpers/emoji * type: fix discord guild type * feat: DiscordEditAutomoderationRule type * fix: remove unused type * feat: DiscordCreateGuildEmoji DiscordModifyGuildEmoji & DiscordModifyCHannel types * feat: DiscordCreateChannel DiscordBuDeleteMessages DiscordCreateMessage DiscordEditMessage * feat: DiscordCreateScheduledEvent EditScheduledEvent DiscordCreateInvite * fix: types for guild stuff for rest * feat: thread discord types for rest * feat: channel rest types * feat: member rest types * feat: more discord rest types * fix: type errors * fix: docs bot param name should be rest * type: fix type error * ci(fix): codecov * chore(client) :add export transformer * fix: verifySignature * test: fix types * test: add test prevent #2683 #2678 * fix: export transformer twice * ci: reuse cache * test: add test:unit-noTextCoverage * feat: add transform and constant package * fix: half fix #2683 #2688 * fix: #2688 fix all transformer.spec.ts * fix: transformer name * ci: update style * fix: dependencies * fix: naming * fix: yarn lock * fix: remove validations from helpers. Closes #2700 * fix: rest routes as a constants pkg * fix: esm import with .js * test: add exception case * feat: adding transformer * chore: change script name * chore: add path fixing to coverage file * fix: camelize tuple bug * fix: reverse transformers * fix: transformers folder to camel * fix: transformers as an object * fix: linter error * ci: run test with deno close #2701 * fix: test depends on build * fix: deno import node:crypto * fix: rest improvements * fix: channel type * fix: export user * chore: move to unit dir * test: moving bench * fix: remove transformers from rest package * fix: move toggle transformers to bot pkg * fix: move out gw helpers to gw package * test: add rest e2e test * fix: syntex * ci: add discord token * fix: ci not passing secret * test: add role test * ci: fix secret inherit * fix: role helpers transformer * test: increase timeout to 10s * fix: member helpers transformer * test: add member test * fix: name and type * fix: guild ban * test: fix test * fix: test add await * test: skip some test * test: increase timeout * chore: add transformer to import map * fix: test * test: add why-is-node-still-running * test: add debug hook * ci: add timeout incase any async running * test: try fix * test: turn on rest debug * fix: if undefined * test: reduce timeout * fix: queue not running after some request * fix: increase remaining after request without ratelimit header * fix: partial webhook * fix: fetch hooks not working if debug defined later * fix: nickname null to undefined * test: finish adding webhook test * refactor(test): move rest to utils * fix: add await * fix: sticker * feat: add message embed component transformer * fix: test not done * fix: arg type to bigString * fix: sendMessage * fix: add allowedMentions, interactionResponse * test: add emoji e2e test * test: add guild e2e test * refactor(test): remove extra rest call * fix: create emoji BINARY_TYPE_INVALID_DATA_URI * fix: create guild rate limit * fix: automode rule helpers and test * fix: test run in only * test: add some queue bucket test * test: remove empty test * refactor(test): use new guild * test: use new guild for other test * fix: guild not defind * reactor(test): remove duplicated creat channel * test: increase timeout to 30s * test: add thread test * fix: more transformers * fix: gateway helpers use transformers * fix: types belong in types pkg * fix: helpers use transformers * BREAKING: v19 rewrite to node + major improvements (#2703) * fix: move all to old folder * fix: cleanup types * fix: more cleanup * fix: more base cleanup * fix: token dotenv * fix: add base transformer * fix: partial error handling * fix: handle 429 rate limit * test(rest): fix unit test * fix(script): transform extension * test: fix error and buffer * feat: camelizer util * fix: cleanup * fix: rate limit queues and headers processing * fix: rest exports * fix: no more transformers * fix: queue header null bug * fix: add gateway package base * fix: lint error * fix: add prettier file * fix: prettier is default fmtr * fix: fmt shard file * fix: fmt * fix: types issue * fix: remove unused consts * fix: all import issues * fix: import error * fix: import ending with .js * fix: remove transformers package * Fix eslint (#2710) * fix: typing of button component label to be optional (#2708) * feat: add guild_connections to role tags and toggles (#2706) Co-authored-by: Skillz4Killz <23035000+Skillz4Killz@users.noreply.github.com> * Add lint checks and autofixing workflow (#2702) * Add lint checks and autofixing workflow * Update lint.yml * Fix: use yarn instead npm * fix: add ts to eslint_extensions * fix: update dir * fix: lint.yml format Co-authored-by: Jonathan Ho <heiheiho000@gmail.com> * fix: unused deps * ci: fix e2e test not running * chore: run yarn install * test: this should run test * feat: getCHannel * feat: createEMji helper * feat: collection class in util package * fix: gateway bugs * why on earth is this needed change * fix: cleanup docs on collection * Emoji rest methods (node-migration-clean) (#2713) * feat: all emoji rest methods * Fix code style issues with ESLint Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * fix: stuff * fix: bot types * fix: remove logs * fix: camelize all gateway payloads * fix: remove todos * fix: start deris * fix: lint/ts errors except shard file * thats 1 way to fix type errors * yes (#2714) Co-authored-by: Skillz4Killz <23035000+Skillz4Killz@users.noreply.github.com> * feat: rest channel helpers * Update release.yml * Update release.yml * ci: add build filter * ci: fail fast false * fix: complete webhook related helpers * fix: follow announcement helper * add forum helper * add stage helpers * add thread related helpers * alphabetize * cleanup webhook routes * automod helpers * scheduled events * integrations helpers * invite stuff incomplete * dm channel and avatar url * chore: move ts-node into package * fix: importing esm * fix: tris message helpers * test(rest): add simplifyUrl test * test(rest): add checkRateLimits test * test(rest): add processRateLimitedPaths test * test(rest): fix missing beforeEach * test(utils): enable old test * interaction helpers * perf(utils): optimize snake to camel case conversion (#2717) * perf(utils): optimize snake to camel case conversion * fmt * wont change much but still faster * actually this was a stupid idea * shh * Fix(client): Fix typings. (#2716) Co-authored-by: Skillz4Killz <23035000+Skillz4Killz@users.noreply.github.com> * test: fix rest and utils test (#2723) * test(utils): fix cant import collection * test(rest): await expect * fix(utils): deno compactability * test(utils): typing * fix(utils): add return type * Add rest helpers for templates (#2727) * Add rest helpers for templates * Fix code style issues with ESLint Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * fix: remove frozenAt * fix: cleanup templates * ci: release to npm (#2725) * ci: release to npm * Update release.yml * 🐛 - fix: types - slashcommands - add nsfw prop (#2731) * feat(rest helpers): Add template and member helpers. (#2728) * feat(rest helpers): Add template and member helpers. * format code * feat(rest): add template routes * fix(rest): routes and runMethod * fix(rest): try to fix most of type Co-authored-by: H01001000 <heiheiho000@gmail.com> Co-authored-by: Skillz4Killz <23035000+Skillz4Killz@users.noreply.github.com> * ci: fix release (#2732) * fix: rest type errors * fix: v19 begin * fix: yarn lock * chore: fix deps and script (#2733) * fix: bug camelizer deleting letters * fix: falsy token check * fix: add frozenat check for queue * fix: max stack trace error do to infinite loop * fix: type error * test(rest): fix missing import (#2734) Co-authored-by: Skillz4Killz <23035000+Skillz4Killz@users.noreply.github.com> * fix: file paths for imports * fix: lastShardId should default to 0 * fix: use isomorphic ws * test(rest): fix "TypeError: [Function] is not a thenable" (#2736) * test(utils): add utils tests (#2737) * test(utils): add urltobase64 test * test(utils): add token test * test(utils): fix missing import buffer * test(utils): add casting test * test(utils): add casting test * test(utils): fix use correct function * chore: make eslint error if missing .js extension (#2735) * chore: eslint error if missing .js extension * chore * lint: fix missing .js error Co-authored-by: Skillz4Killz <23035000+Skillz4Killz@users.noreply.github.com> * fix: readme runtime list (#2739) * Bot pkg (#2740) * fix: bot pkg test * Fix code style issues with ESLint * Update Guild.ts * Update Guild.ts * Fix code style issues with ESLint Co-authored-by: Lint Action <lint-action@samuelmeuli.com> Co-authored-by: Jonathan Ho <heiheiho000@gmail.com> * Update tsdoc.json (#2741) Updates tsdoc.json to reflect the current packages. * Interaction types - remove member,channel,role from value type (#2743) https://discord.com/channels/785384884197392384/1067265182776176690/1068189883073572924 * Add missing types (#2742) * Revert "fix: use isomorphic ws" (#2744) This reverts commitad306b0d0a. * fix: interaction requests that sent without full url * fix: lint issues * fix: remote gateway test file * fix: interaction response bug with body being invalid (#2746) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * fix: interaction followup type (#2747) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests * fix: perma fix for type error in ci * fix: followupmessage option type --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * fix: e2e exit bug (#2748) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests * fix: perma fix for type error in ci * fix: followupmessage option type * fix: e2e tests exit bug * fix: color console logger --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * fix: rest sending attachments files (#2749) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests * fix: perma fix for type error in ci * fix: followupmessage option type * fix: e2e tests exit bug * fix: color console logger * fix: image file sending * Fix code style issues with ESLint --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * fix: guild and role methods (#2751) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests * fix: perma fix for type error in ci * fix: followupmessage option type * fix: e2e tests exit bug * fix: color console logger * fix: image file sending * Fix code style issues with ESLint * guild and role methods * Fix code style issues with ESLint --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * ci: add bot package (#2752) * ci: add bot package * ci: fix test * e2e test stuff (#2754) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests * fix: perma fix for type error in ci * fix: followupmessage option type * fix: e2e tests exit bug * fix: color console logger * fix: image file sending * Fix code style issues with ESLint * guild and role methods * Fix code style issues with ESLint * fix: dont send heartbeat if socket is not open * fix: remove logs * fox: remove more logs * fix some bugs in role tests * Switch to after hook style * hoti's speed snaker * auto convert objects for discord * Fix code style issues with ESLint * fix: remove dup imports * fix: i hate linters * speeder --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * fix: delete guilds test (#2758) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests * fix: perma fix for type error in ci * fix: followupmessage option type * fix: e2e tests exit bug * fix: color console logger * fix: image file sending * Fix code style issues with ESLint * guild and role methods * Fix code style issues with ESLint * fix: dont send heartbeat if socket is not open * fix: remove logs * fox: remove more logs * fix some bugs in role tests * Switch to after hook style * hoti's speed snaker * auto convert objects for discord * Fix code style issues with ESLint * fix: remove dup imports * fix: i hate linters * speeder * fix: tests delete guilds * Fix code style issues with ESLint --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * chore: fix deps (#2757) * ci/test: fix bot pkg e2e test (#2759) * chore: fix script and update import map (#2761) * chore: fix version script * chore: update import map * chore: fix ws import map * chore: fix deno import map * test: fix * fix: more rest e2e tests (#2763) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests * fix: perma fix for type error in ci * fix: followupmessage option type * fix: e2e tests exit bug * fix: color console logger * fix: image file sending * Fix code style issues with ESLint * guild and role methods * Fix code style issues with ESLint * fix: dont send heartbeat if socket is not open * fix: remove logs * fox: remove more logs * fix some bugs in role tests * Switch to after hook style * hoti's speed snaker * auto convert objects for discord * Fix code style issues with ESLint * fix: remove dup imports * fix: i hate linters * speeder * fix: tests delete guilds * Fix code style issues with ESLint * fix: easier to provide custom intents in bot * fix: shutdown bot after test * fix: add getGuild * fix: multiple guild delete attempts * fix: add emoji e2e tests * fix: remaining old e2e rest tests * Fix code style issues with ESLint --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * test: add gateway integration test (#2756) * test: add gateway integration test * test(gateway): fix connection test * test(gateway): add heartbeat test * ci: add integration test * fix: add uWebSockets.js * ci: add timeout * test(utils): remove old test * Revert "test(utils): remove old test" This reverts commit04fb6dd4b5. * test(gateway): fix uws server * test(gateway): fix type * chore: update codecov flag * test(gateway): remove dev code --------- Co-authored-by: Skillz4Killz <23035000+Skillz4Killz@users.noreply.github.com> * Update release.yml (#2768) * fix: bot logger (#2769) * fix: readme runtime list * Fix code style issues with ESLint * node 18 * fix: websocket import type * fix: body for interaction requests * fix: perma fix for type error in ci * fix: followupmessage option type * fix: e2e tests exit bug * fix: color console logger * fix: image file sending * Fix code style issues with ESLint * guild and role methods * Fix code style issues with ESLint * fix: dont send heartbeat if socket is not open * fix: remove logs * fox: remove more logs * fix some bugs in role tests * Switch to after hook style * hoti's speed snaker * auto convert objects for discord * Fix code style issues with ESLint * fix: remove dup imports * fix: i hate linters * speeder * fix: tests delete guilds * Fix code style issues with ESLint * fix: easier to provide custom intents in bot * fix: shutdown bot after test * fix: add getGuild * fix: multiple guild delete attempts * fix: add emoji e2e tests * fix: remaining old e2e rest tests * Fix code style issues with ESLint * fix: add bot.logger * fix: make logger name capital --------- Co-authored-by: Lint Action <lint-action@samuelmeuli.com> * chore: update readme (#2772) * chore: add coverage per pkg * chore: add npm version * chore: add test status * chore: fix ci * chore: fix ci * ci: fix ci needs * chore: add only push event * style: remove import logger * 📚 - docs: fix README package links (#2773) * test(all): add test importing index (#2774) * test(all): add test importing index * chore: remove old benchmark dir * chore: disable coverage status fail * test(client): add import test with try catch * test(rest): fix narrow import scope * test(utils): add test (#2764) * test(utils): remove old test * test(utils): add color test * test(utils): fix import mocha * test(utils): fix test type error * test(utils): remove dev code * fix(utils): bucket not export all function * test(utils): add some test for bucket * fix(utils): close #2775 * test(utils): add test for permissions.ts * test(utils): fix missing mocha import * fix(utils): better fix for #2775 * feat: addReaction & addReactions * feat: connectToVoice * fix: linters issues * fix: remove aliases and add createGuildFromTemplate * feat: deleteMessages * fix: reaction related helpers * mfa level * voie states editing * image urls * fix: typos * get message typeguards * fix: more helpers * fix: remaining helpers * fix: add logs to gateway manager * fix: rest resolve sends status and body * fix: lots of errors * fix: client errors * fix: remove old pkg * snaker * fix: broken util import for image url * fix: cleanup shard and circular deps * fix: remove ThreadChannel from GuildChannel * fix: generate interaction usage * fix: more bugs * fix: use node:events to import * fix(rest): add interface RestRequestRejection (#2782) * fix: remove invalid todo * fix: timeout bug --------- Co-authored-by: Skillz <skillz@discord.gg/ddeno> Co-authored-by: Jonathan Ho <heiheiho000@gmail.com> Co-authored-by: deepsarda <92147339+deepsarda@users.noreply.github.com> Co-authored-by: Awesome Stickz <awesome@stickz.dev> Co-authored-by: Lint Action <lint-action@samuelmeuli.com> Co-authored-by: Lars_und_so <46791248+Larsundso@users.noreply.github.com> Co-authored-by: ITOH <to@itoh.at> Co-authored-by: Yaikava <83710104+Yaikava@users.noreply.github.com> Co-authored-by: Andreas Fink <mail@afink.dev> --------- Co-authored-by: Jonathan Ho <48591478+H01001000@users.noreply.github.com> Co-authored-by: H01001000 <heiheiho000@gmail.com> Co-authored-by: deepsarda <92147339+deepsarda@users.noreply.github.com> Co-authored-by: Yaikava <83710104+Yaikava@users.noreply.github.com> Co-authored-by: Skillz <skillz@discord.gg/ddeno> Co-authored-by: Awesome Stickz <awesome@stickz.dev> Co-authored-by: Lint Action <lint-action@samuelmeuli.com> Co-authored-by: Lars_und_so <46791248+Larsundso@users.noreply.github.com> Co-authored-by: ITOH <to@itoh.at> Co-authored-by: Andreas Fink <mail@afink.dev>
This commit is contained in:
Skillz4Killz
co-authored by
H01001000
Lint Action
Skillz
deepsarda
Awesome Stickz
Lars_und_so
ITOH
Yaikava
Andreas Fink
Jonathan Ho
H01001000
Lint Action
Skillz
deepsarda
Awesome Stickz
Lars_und_so
ITOH
Yaikava
Andreas Fink
Jonathan Ho
parent
1a717ddc7d
commit
2714e1e7d1
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"all": true,
|
||||
"src": "src",
|
||||
"reporter": ["text", "lcov"],
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["tests"]
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"require": "ts-node/register",
|
||||
"loader": "ts-node/esm",
|
||||
"recursive": true,
|
||||
"timeout": 2000,
|
||||
"watch-extensions": "ts",
|
||||
"watch-files": ["src", "tests"],
|
||||
"enable-source-maps": true,
|
||||
"parallel": false
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"minify": true,
|
||||
"jsc": {
|
||||
"parser": {
|
||||
"syntax": "typescript",
|
||||
"decorators": true,
|
||||
"dynamicImport": true
|
||||
},
|
||||
"transform": {
|
||||
"legacyDecorator": true,
|
||||
"decoratorMetadata": true
|
||||
},
|
||||
"target": "es2022",
|
||||
"keepClassNames": true,
|
||||
"loose": true,
|
||||
"minify": {
|
||||
"compress": {
|
||||
"unused": true
|
||||
},
|
||||
"mangle": true
|
||||
}
|
||||
},
|
||||
"module": {
|
||||
"type": "es6",
|
||||
"strict": false,
|
||||
"strictMode": true,
|
||||
"lazy": false,
|
||||
"noInterop": false
|
||||
},
|
||||
"sourceMaps": "inline"
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
# Standalone WS / Proxy WS
|
||||
|
||||
This WS service is meant for ADVANCED DEVELOPERS ONLY!
|
||||
|
||||
## Benefits
|
||||
|
||||
- **Zero Downtime Updates**:
|
||||
|
||||
- Your bot can be updated in a matter of seconds. With normal sharding, you have to restart which also has to process
|
||||
identifying all your shards with a 1/~5s rate limit. With WS handling moved to a proxy process, this allows you to
|
||||
instantly get the bot code restarted without any concerns of delays. If you have a bot on 200,000 servers normally
|
||||
this would mean a 20 minute delay to restart your bot if you made a small change and restarted.
|
||||
|
||||
- **Zero Downtime Resharding**:
|
||||
|
||||
- Discord stops letting your bot get added to new servers at certain points in time. For example, suppose you had
|
||||
150,000 servers running 150 shards. The maximum amount of servers your shards could hold is 150 \* 2500 = 375,000.
|
||||
If your bot reaches this, it can no longer join new servers until it re-shards.
|
||||
- DD proxy provides 2 types of re-sharding. Automated and manual. You can also have both.
|
||||
- `Automated`: This system will automatically begin a Zero-downtime resharding process behind the scenes when you
|
||||
reach 80% of your maximum servers allowed by your shards. For example, since 375,000 was the max, at 300,000 we
|
||||
would begin re-sharding behind the scenes with `ZERO DOWNTIME`.
|
||||
- 80% of maximum servers reached (The % of 80% is customizable.)
|
||||
- Identify limits have room to allow re-sharding. (Also customizable)
|
||||
- `Manual`: You can also trigger this manually should you choose.
|
||||
|
||||
- **Horizontal Scaling**:
|
||||
|
||||
- The proxy system allows you to scale the bot horizontally. When you reach a huge size, you can either keep spending
|
||||
more money to keep beefing up your server or you can buy several cheaper servers and scale horizontally. The proxy
|
||||
means you can have WS handling on a completely separate system.
|
||||
|
||||
- **No Loss Restarts**:
|
||||
|
||||
- When you restart a bot without the proxy system, normally you would lose many events. Users may be using commands or
|
||||
messages are sent that will not be filtered. As your bot's grow this number rises dramatically. Users may join who
|
||||
wont get the auto-roles or any other actions your bot should take. With the proxy system, you can keep restarting
|
||||
your bot and never lose any events. Events will be put into a queue while your bot is down(max size of queue is
|
||||
customizable), once the bot is available the queue will begin processing all events.
|
||||
|
||||
- **Controllers**:
|
||||
|
||||
- The controller aspect gives you full control over everything inside the proxy. You can provide a function to simply
|
||||
override the handler. For example, if you would like a certain function to do something different, instead of having
|
||||
to fork and maintain your fork, you can just provide a function to override.
|
||||
|
||||
- **Clustering With Workers**:
|
||||
- Take full advantage of all your CPU cores by using workers to spread the load. Control how many shards per worker
|
||||
and how many workers to maximize efficiency!
|
||||
|
||||
## Usage
|
||||
|
||||
```ts
|
||||
createGatewayManager({
|
||||
// TODO: (docs) Fill this out
|
||||
});
|
||||
```
|
||||
|
||||
## API/Docs
|
||||
|
||||
// TODO: (docs) Fill this out. List all props/methods.
|
||||
@@ -0,0 +1,50 @@
|
||||
{
|
||||
"name": "@discordeno/gateway",
|
||||
"version": "19.0.0-alpha.1",
|
||||
"main": "./dist/index.js",
|
||||
"types": "./dist/index.d.ts",
|
||||
"type": "module",
|
||||
"license": "Apache-2.0",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/discordeno/discordeno.git"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "swc --delete-dir-on-start src --out-dir dist",
|
||||
"build:type": "tsc --declaration --emitDeclarationOnly --declarationDir dist",
|
||||
"release-build": "yarn build && yarn build:type",
|
||||
"fmt": "eslint --fix \"src/**/*.ts*\"",
|
||||
"lint": "eslint \"src/**/*.ts*\"",
|
||||
"test:unit-coverage": "c8 mocha --no-warnings 'tests/unit/**/*.spec.ts'",
|
||||
"test:unit": "c8 --r lcov mocha --no-warnings 'tests/unit/**/*.spec.ts' && node ../../scripts/coveragePathFixing.js gateway",
|
||||
"test:deno-unit": "swc tests --delete-dir-on-start --out-dir denoTestsDist && node ../../scripts/fixDenoTestExtension.js && deno test -A --import-map ../../denoImportMap.json denoTestsDist/unit",
|
||||
"test:unit:watch": "mocha --no-warnings --watch --parallel 'tests/unit/**/*.spec.ts'",
|
||||
"test:integration": "c8 --r lcov mocha --no-warnings 'tests/integration/**/*.spec.ts' && node ../../scripts/coveragePathFixing.js gateway",
|
||||
"test:type": "tsc --noEmit",
|
||||
"test:test-type": "tsc --project tsconfig.test.json"
|
||||
},
|
||||
"dependencies": {
|
||||
"@discordeno/types": "19.0.0-alpha.1",
|
||||
"@discordeno/utils": "19.0.0-alpha.1",
|
||||
"ws": "^8.11.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@swc/cli": "^0.1.57",
|
||||
"@swc/core": "^1.3.21",
|
||||
"@types/chai": "^4",
|
||||
"@types/mocha": "^10",
|
||||
"@types/node": "^18.11.9",
|
||||
"@types/sinon": "^10.0.13",
|
||||
"@types/ws": "^8.5.3",
|
||||
"c8": "^7.12.0",
|
||||
"chai": "^4.3.7",
|
||||
"eslint": "^8.0.1",
|
||||
"eslint-config-discordeno": "*",
|
||||
"mocha": "^10.1.0",
|
||||
"sinon": "^15.0.0",
|
||||
"ts-node": "^10.9.1",
|
||||
"tsconfig": "*",
|
||||
"typescript": "^4.9.3",
|
||||
"uWebSockets.js": "uNetworking/uWebSockets.js#v20.19.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,753 @@
|
||||
import type {
|
||||
AtLeastOne,
|
||||
BigString,
|
||||
Camelize,
|
||||
DiscordGatewayPayload,
|
||||
DiscordHello,
|
||||
DiscordMember,
|
||||
DiscordReady,
|
||||
RequestGuildMembers,
|
||||
} from '@discordeno/types'
|
||||
import { GatewayCloseEventCodes, GatewayIntents, GatewayOpcodes } from '@discordeno/types'
|
||||
import { camelize, Collection, createLeakyBucket, delay, logger } from '@discordeno/utils'
|
||||
import { inflateSync } from 'node:zlib'
|
||||
import WebSocket from 'ws'
|
||||
import type { RequestMemberRequest } from './manager.js'
|
||||
import type { BotStatusUpdate, ShardEvents, ShardGatewayConfig, ShardHeart, ShardSocketRequest, StatusUpdate, UpdateVoiceState } from './types.js'
|
||||
import { ShardSocketCloseCodes, ShardState } from './types.js'
|
||||
|
||||
export class Shard {
|
||||
/** The id of the shard */
|
||||
id: number
|
||||
/** The connection config details that this shard will used to connect to discord. */
|
||||
connection: ShardGatewayConfig
|
||||
/** This contains all the heartbeat information */
|
||||
heart: ShardHeart
|
||||
/** The maximum of requests which can be send to discord per rate limit tick. Typically this value should not be changed. */
|
||||
maxRequestsPerRateLimitTick: number = 120
|
||||
/** The previous payload sequence number. */
|
||||
previousSequenceNumber: number | null = null
|
||||
/** In which interval (in milliseconds) the gateway resets it's rate limit. */
|
||||
rateLimitResetInterval: number = 60000
|
||||
/** Current session id of the shard if present. */
|
||||
sessionId?: string
|
||||
/** This contains the WebSocket connection to Discord, if currently connected. */
|
||||
socket?: WebSocket
|
||||
/** Current internal state of the this. */
|
||||
state = ShardState.Offline
|
||||
/** The url provided by discord to use when resuming a connection for this this. */
|
||||
resumeGatewayUrl: string = ''
|
||||
/** The shard related event handlers. */
|
||||
events: ShardEvents = {}
|
||||
/** Cache for pending gateway requests which should have been send while the gateway went offline. */
|
||||
offlineSendQueue: Array<(_?: unknown) => void> = []
|
||||
/** Resolve internal waiting states. Mapped by SelectedEvents => ResolveFunction */
|
||||
resolves = new Map<'READY' | 'RESUMED' | 'INVALID_SESSION', (payload: DiscordGatewayPayload) => void>()
|
||||
/** Shard bucket. Only access this if you know what you are doing. Bucket for handling shard request rate limits. */
|
||||
bucket = createLeakyBucket({
|
||||
max: 120,
|
||||
refillInterval: 60000,
|
||||
refillAmount: 120,
|
||||
})
|
||||
|
||||
/** This managers cache related settings. */
|
||||
cache = {
|
||||
requestMembers: {
|
||||
/**
|
||||
* Whether or not request member requests should be cached.
|
||||
* @default false
|
||||
*/
|
||||
enabled: false,
|
||||
/** The pending requests. */
|
||||
pending: new Collection<string, RequestMemberRequest>(),
|
||||
},
|
||||
}
|
||||
|
||||
constructor(options: ShardCreateOptions) {
|
||||
this.id = options.id
|
||||
this.connection = options.connection
|
||||
this.events = options.events
|
||||
|
||||
this.heart = {
|
||||
acknowledged: false,
|
||||
interval: 45000,
|
||||
}
|
||||
}
|
||||
|
||||
/** The gateway configuration which is used to connect to Discord. */
|
||||
get gatewayConfig(): ShardGatewayConfig {
|
||||
return this.connection
|
||||
}
|
||||
|
||||
/** Calculate the amount of requests which can safely be made per rate limit interval, before the gateway gets disconnected due to an exceeded rate limit. */
|
||||
calculateSafeRequests(): number {
|
||||
// * 2 adds extra safety layer for discords OP 1 requests that we need to respond to
|
||||
const safeRequests = this.maxRequestsPerRateLimitTick - Math.ceil(this.rateLimitResetInterval / this.heart.interval) * 2
|
||||
|
||||
return safeRequests < 0 ? 0 : safeRequests
|
||||
}
|
||||
|
||||
async checkOffline(highPriority: boolean): Promise<void> {
|
||||
if (!this.isOpen()) {
|
||||
await new Promise((resolve) => {
|
||||
// Higher priority requests get added at the beginning of the array.
|
||||
if (highPriority) this.offlineSendQueue.unshift(resolve)
|
||||
else this.offlineSendQueue.push(resolve)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/** Close the socket connection to discord if present. */
|
||||
close(code: number, reason: string): void {
|
||||
if (this.socket?.readyState !== WebSocket.OPEN) return
|
||||
|
||||
return this.socket?.close(code, reason)
|
||||
}
|
||||
|
||||
/** Connect the shard with the gateway and start heartbeating. This will not identify the shard to the gateway. */
|
||||
async connect(): Promise<Shard> {
|
||||
// Only set the shard to `Connecting` state,
|
||||
// if the connection request does not come from an identify or resume action.
|
||||
if (![ShardState.Identifying, ShardState.Resuming].includes(this.state)) {
|
||||
this.state = ShardState.Connecting
|
||||
}
|
||||
this.events.connecting?.(this)
|
||||
|
||||
let url = new URL(this.gatewayConfig.url)
|
||||
// If not connecting to a proxy but directly to discord need to handle resuming
|
||||
if (url.origin === 'wss://gateway.discord.gg') {
|
||||
if (this.state === ShardState.Resuming) {
|
||||
url = new URL(this.resumeGatewayUrl)
|
||||
}
|
||||
url.searchParams.set('v', this.gatewayConfig.version.toString())
|
||||
url.searchParams.set('encoding', 'json')
|
||||
}
|
||||
|
||||
const socket = new WebSocket(url.toString())
|
||||
|
||||
this.socket = socket
|
||||
|
||||
// TODO: proper event handling
|
||||
socket.onerror = (event) => console.log({ error: event })
|
||||
|
||||
socket.onclose = async (event) => await this.handleClose(event)
|
||||
|
||||
socket.onmessage = async (message) => await this.handleMessage(message)
|
||||
|
||||
return await new Promise((resolve) => {
|
||||
socket.onopen = () => {
|
||||
// Only set the shard to `Unidentified` state,
|
||||
// if the connection request does not come from an identify or resume action.
|
||||
if (![ShardState.Identifying, ShardState.Resuming].includes(this.state)) {
|
||||
this.state = ShardState.Unidentified
|
||||
}
|
||||
this.events.connected?.(this)
|
||||
|
||||
resolve(this)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/** Identify the shard to the gateway. If not connected, this will also connect the shard to the gateway. */
|
||||
async identify(): Promise<void> {
|
||||
// A new identify has been requested even though there is already a connection open.
|
||||
// Therefore we need to close the old connection and heartbeating before creating a new one.
|
||||
if (this.isOpen()) {
|
||||
console.log(`CLOSING EXISTING SHARD: #${this.id}`)
|
||||
this.close(ShardSocketCloseCodes.ReIdentifying, 'Re-identifying closure of old connection.')
|
||||
}
|
||||
|
||||
this.state = ShardState.Identifying
|
||||
this.events.identifying?.(this)
|
||||
|
||||
// It is possible that the shard is in Heartbeating state but not identified,
|
||||
// so check whether there is already a gateway connection existing.
|
||||
// If not we need to create one before we identify.
|
||||
if (!this.isOpen()) {
|
||||
await this.connect()
|
||||
}
|
||||
|
||||
// Wait until an identify is free for this this.
|
||||
await this.requestIdentify()
|
||||
|
||||
this.send(
|
||||
{
|
||||
op: GatewayOpcodes.Identify,
|
||||
d: {
|
||||
token: `Bot ${this.gatewayConfig.token}`,
|
||||
compress: this.gatewayConfig.compress,
|
||||
properties: this.gatewayConfig.properties,
|
||||
intents: this.gatewayConfig.intents,
|
||||
shard: [this.id, this.gatewayConfig.totalShards],
|
||||
presence: await this.makePresence?.(),
|
||||
},
|
||||
},
|
||||
true,
|
||||
)
|
||||
|
||||
return await new Promise((resolve) => {
|
||||
this.resolves.set('READY', () => {
|
||||
this.events.identified?.(this)
|
||||
resolve()
|
||||
})
|
||||
// When identifying too fast,
|
||||
// Discord sends an invalid session payload.
|
||||
// This can safely be ignored though and the shard starts a new identify action.
|
||||
this.resolves.set('INVALID_SESSION', () => {
|
||||
this.resolves.delete('READY')
|
||||
resolve()
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/** Check whether the connection to Discord is currently open. */
|
||||
isOpen(): boolean {
|
||||
return this.socket?.readyState === WebSocket.OPEN
|
||||
}
|
||||
|
||||
/** Attempt to resume the previous shards session with the gateway. */
|
||||
async resume(): Promise<void> {
|
||||
// gateway.debug("GW RESUMING", { shardId });
|
||||
// It has been requested to resume the Shards session.
|
||||
// It's possible that the shard is still connected with Discord's gateway therefore we need to forcefully close it.
|
||||
if (this.isOpen()) {
|
||||
this.close(ShardSocketCloseCodes.ResumeClosingOldConnection, 'Reconnecting the shard, closing old connection.')
|
||||
}
|
||||
|
||||
// Shard has never identified, so we cannot resume.
|
||||
if (!this.sessionId) {
|
||||
// gateway.debug(
|
||||
// "GW DEBUG",
|
||||
// `[Error] Trying to resume a shard (id: ${shardId}) that was not first identified.`,
|
||||
// );
|
||||
|
||||
return await this.identify()
|
||||
|
||||
// throw new Error(`[SHARD] Trying to resume a shard (id: ${this.id}) which was never identified`);
|
||||
}
|
||||
|
||||
this.state = ShardState.Resuming
|
||||
|
||||
// Before we can resume, we need to create a new connection with Discord's gateway.
|
||||
await this.connect()
|
||||
|
||||
this.send(
|
||||
{
|
||||
op: GatewayOpcodes.Resume,
|
||||
d: {
|
||||
token: `Bot ${this.gatewayConfig.token}`,
|
||||
session_id: this.sessionId,
|
||||
seq: this.previousSequenceNumber ?? 0,
|
||||
},
|
||||
},
|
||||
true,
|
||||
)
|
||||
|
||||
return await new Promise((resolve) => {
|
||||
this.resolves.set('RESUMED', () => resolve())
|
||||
// If it is attempted to resume with an invalid session id,
|
||||
// Discord sends an invalid session payload
|
||||
// Not erroring here since it is easy that this happens, also it would be not catchable
|
||||
this.resolves.set('INVALID_SESSION', () => {
|
||||
this.resolves.delete('RESUMED')
|
||||
resolve()
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/** Send a message to Discord.
|
||||
* @param {boolean} [highPriority=false] - Whether this message should be send asap.
|
||||
*/
|
||||
async send(message: ShardSocketRequest, highPriority = false): Promise<void> {
|
||||
// Before acquiring a token from the bucket, check whether the shard is currently offline or not.
|
||||
// Else bucket and token wait time just get wasted.
|
||||
await this.checkOffline(highPriority)
|
||||
|
||||
await this.bucket.acquire(1, highPriority)
|
||||
|
||||
// It's possible, that the shard went offline after a token has been acquired from the bucket.
|
||||
await this.checkOffline(highPriority)
|
||||
|
||||
this.socket?.send(JSON.stringify(message))
|
||||
}
|
||||
|
||||
/** Shutdown the this. Forcefully disconnect the shard from Discord. The shard may not attempt to reconnect with Discord. */
|
||||
async shutdown(): Promise<void> {
|
||||
this.close(ShardSocketCloseCodes.Shutdown, 'Shard shutting down.')
|
||||
this.state = ShardState.Offline
|
||||
}
|
||||
|
||||
/** Handle a gateway connection close. */
|
||||
async handleClose(close: WebSocket.CloseEvent): Promise<void> {
|
||||
// gateway.debug("GW CLOSED", { shardId, payload: event });
|
||||
|
||||
this.stopHeartbeating()
|
||||
|
||||
switch (close.code) {
|
||||
case ShardSocketCloseCodes.TestingFinished: {
|
||||
this.state = ShardState.Offline
|
||||
this.events.disconnected?.(this)
|
||||
|
||||
return
|
||||
}
|
||||
// On these codes a manual start will be done.
|
||||
case ShardSocketCloseCodes.Shutdown:
|
||||
case ShardSocketCloseCodes.ReIdentifying:
|
||||
case ShardSocketCloseCodes.Resharded:
|
||||
case ShardSocketCloseCodes.ResumeClosingOldConnection:
|
||||
case ShardSocketCloseCodes.ZombiedConnection: {
|
||||
this.state = ShardState.Disconnected
|
||||
this.events.disconnected?.(this)
|
||||
|
||||
// gateway.debug("GW CLOSED_RECONNECT", { shardId, payload: event });
|
||||
return
|
||||
}
|
||||
// Gateway connection closes which require a new identify.
|
||||
case GatewayCloseEventCodes.UnknownOpcode:
|
||||
case GatewayCloseEventCodes.NotAuthenticated:
|
||||
case GatewayCloseEventCodes.InvalidSeq:
|
||||
case GatewayCloseEventCodes.RateLimited:
|
||||
case GatewayCloseEventCodes.SessionTimedOut: {
|
||||
this.state = ShardState.Identifying
|
||||
this.events.disconnected?.(this)
|
||||
|
||||
return await this.identify()
|
||||
}
|
||||
// When these codes are received something went really wrong.
|
||||
// On those we cannot start a reconnect attempt.
|
||||
case GatewayCloseEventCodes.AuthenticationFailed:
|
||||
case GatewayCloseEventCodes.InvalidShard:
|
||||
case GatewayCloseEventCodes.ShardingRequired:
|
||||
case GatewayCloseEventCodes.InvalidApiVersion:
|
||||
case GatewayCloseEventCodes.InvalidIntents:
|
||||
case GatewayCloseEventCodes.DisallowedIntents: {
|
||||
this.state = ShardState.Offline
|
||||
this.events.disconnected?.(this)
|
||||
|
||||
throw new Error(close.reason || 'Discord gave no reason! GG! You broke Discord!')
|
||||
}
|
||||
// Gateway connection closes on which a resume is allowed.
|
||||
case GatewayCloseEventCodes.UnknownError:
|
||||
case GatewayCloseEventCodes.DecodeError:
|
||||
case GatewayCloseEventCodes.AlreadyAuthenticated:
|
||||
default: {
|
||||
this.state = ShardState.Resuming
|
||||
this.events.disconnected?.(this)
|
||||
|
||||
return await this.resume()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Handles a incoming gateway packet. */
|
||||
async handleDiscordPacket(packet: DiscordGatewayPayload): Promise<void> {
|
||||
// Edge case start: https://github.com/discordeno/discordeno/issues/2311
|
||||
this.heart.lastAck = Date.now()
|
||||
// Manually calculating the round trip time for users who need it.
|
||||
if (this.heart.lastBeat && !this.heart.acknowledged) {
|
||||
this.heart.rtt = this.heart.lastAck - this.heart.lastBeat
|
||||
}
|
||||
this.heart.acknowledged = true
|
||||
// Edge case end!
|
||||
|
||||
switch (packet.op) {
|
||||
case GatewayOpcodes.Heartbeat: {
|
||||
// TODO: can this actually happen
|
||||
if (!this.isOpen()) return
|
||||
|
||||
this.heart.lastBeat = Date.now()
|
||||
// Discord randomly sends this requiring an immediate heartbeat back.
|
||||
// Using a direct socket.send call here because heartbeat requests are reserved by us.
|
||||
this.socket?.send(
|
||||
JSON.stringify({
|
||||
op: GatewayOpcodes.Heartbeat,
|
||||
d: this.previousSequenceNumber,
|
||||
}),
|
||||
)
|
||||
this.events.heartbeat?.(this)
|
||||
|
||||
break
|
||||
}
|
||||
case GatewayOpcodes.Hello: {
|
||||
const interval = (packet.d as DiscordHello).heartbeat_interval
|
||||
|
||||
this.startHeartbeating(interval)
|
||||
|
||||
if (this.state !== ShardState.Resuming) {
|
||||
// HELLO has been send on a non resume action.
|
||||
// This means that the shard starts a new session,
|
||||
// therefore the rate limit interval has been reset too.
|
||||
this.bucket = createLeakyBucket({
|
||||
max: this.calculateSafeRequests(),
|
||||
refillInterval: 60000,
|
||||
refillAmount: this.calculateSafeRequests(),
|
||||
// Waiting acquires should not be lost on a re-identify.
|
||||
waiting: this.bucket.waiting,
|
||||
})
|
||||
}
|
||||
|
||||
this.events.hello?.(this)
|
||||
|
||||
break
|
||||
}
|
||||
case GatewayOpcodes.HeartbeatACK: {
|
||||
this.events.heartbeatAck?.(this)
|
||||
|
||||
break
|
||||
}
|
||||
case GatewayOpcodes.Reconnect: {
|
||||
// gateway.debug("GW RECONNECT", { shardId });
|
||||
|
||||
this.events.requestedReconnect?.(this)
|
||||
|
||||
await this.resume()
|
||||
|
||||
break
|
||||
}
|
||||
case GatewayOpcodes.InvalidSession: {
|
||||
// gateway.debug("GW INVALID_SESSION", { shardId, payload: packet });
|
||||
const resumable = packet.d as boolean
|
||||
|
||||
this.events.invalidSession?.(this, resumable)
|
||||
|
||||
// We need to wait for a random amount of time between 1 and 5
|
||||
// Reference: https://discord.com/developers/docs/topics/gateway#resuming
|
||||
await delay(Math.floor((Math.random() * 4 + 1) * 1000))
|
||||
|
||||
this.resolves.get('INVALID_SESSION')?.(packet)
|
||||
this.resolves.delete('INVALID_SESSION')
|
||||
|
||||
// When resumable is false we need to re-identify
|
||||
if (!resumable) {
|
||||
await this.identify()
|
||||
|
||||
break
|
||||
}
|
||||
|
||||
// The session is invalid but apparently it is resumable
|
||||
await this.resume()
|
||||
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if (packet.t === 'RESUMED') {
|
||||
// gateway.debug("GW RESUMED", { shardId });
|
||||
|
||||
this.state = ShardState.Connected
|
||||
this.events.resumed?.(this)
|
||||
|
||||
// Continue the requests which have been queued since the shard went offline.
|
||||
this.offlineSendQueue.map((resolve) => resolve())
|
||||
|
||||
this.resolves.get('RESUMED')?.(packet)
|
||||
this.resolves.delete('RESUMED')
|
||||
} else if (packet.t === 'READY') {
|
||||
// Important for future resumes.
|
||||
|
||||
const payload = packet.d as DiscordReady
|
||||
|
||||
this.resumeGatewayUrl = payload.resume_gateway_url
|
||||
|
||||
this.sessionId = payload.session_id
|
||||
this.state = ShardState.Connected
|
||||
|
||||
// Continue the requests which have been queued since the shard went offline.
|
||||
// Important when this is a re-identify
|
||||
this.offlineSendQueue.map((resolve) => resolve())
|
||||
|
||||
this.resolves.get('READY')?.(packet)
|
||||
this.resolves.delete('READY')
|
||||
}
|
||||
|
||||
// Update the sequence number if it is present
|
||||
// `s` can be either `null` or a `number`.
|
||||
// In order to prevent update misses when `s` is `0` we check against null.
|
||||
if (packet.s !== null) {
|
||||
this.previousSequenceNumber = packet.s
|
||||
}
|
||||
|
||||
// The necessary handling required for the Shards connection has been finished.
|
||||
// Now the event can be safely forwarded.
|
||||
this.events.message?.(this, camelize(packet))
|
||||
}
|
||||
|
||||
/** Handle an incoming gateway message. */
|
||||
async handleMessage(message: WebSocket.MessageEvent): Promise<void> {
|
||||
let preProcessMessage = message.data
|
||||
|
||||
// If message compression is enabled,
|
||||
// Discord might send zlib compressed payloads.
|
||||
if (this.gatewayConfig.compress && preProcessMessage instanceof Blob) {
|
||||
preProcessMessage = inflateSync(await preProcessMessage.arrayBuffer()).toString()
|
||||
}
|
||||
|
||||
// Safeguard incase decompression failed to make a string.
|
||||
if (typeof preProcessMessage !== 'string') return
|
||||
|
||||
return await this.handleDiscordPacket(JSON.parse(preProcessMessage) as DiscordGatewayPayload)
|
||||
}
|
||||
|
||||
/**
|
||||
* Override in order to make the shards presence.
|
||||
* async in case devs create the presence based on eg. database values.
|
||||
* Passing the shard's id there to make it easier for the dev to use this function.
|
||||
*/
|
||||
async makePresence(): Promise<BotStatusUpdate | undefined> {
|
||||
// eslint-disable-next-line no-useless-return
|
||||
return
|
||||
}
|
||||
|
||||
/** This function communicates with the management process, in order to know whether its free to identify. When this function resolves, this means that the shard is allowed to send an identify payload to discord. */
|
||||
async requestIdentify(): Promise<void> {
|
||||
// TODO: how to handle this
|
||||
// return await options.requestIdentify(this.id)
|
||||
}
|
||||
|
||||
/** Start sending heartbeat payloads to Discord in the provided interval. */
|
||||
startHeartbeating(interval: number): void {
|
||||
// gateway.debug("GW HEARTBEATING_STARTED", { shardId, interval });
|
||||
|
||||
this.heart.interval = interval
|
||||
|
||||
// Only set the shard's state to `Unidentified`
|
||||
// if heartbeating has not been started due to an identify or resume action.
|
||||
if ([ShardState.Disconnected, ShardState.Offline].includes(this.state)) {
|
||||
this.state = ShardState.Unidentified
|
||||
}
|
||||
|
||||
// The first heartbeat needs to be send with a random delay between `0` and `interval`
|
||||
// Using a `setTimeout(_, jitter)` here to accomplish that.
|
||||
// `Math.random()` can be `0` so we use `0.5` if this happens
|
||||
// Reference: https://discord.com/developers/docs/topics/gateway#heartbeating
|
||||
const jitter = Math.ceil(this.heart.interval * (Math.random() || 0.5))
|
||||
this.heart.timeoutId = setTimeout(() => {
|
||||
if (!this.isOpen()) return
|
||||
|
||||
// Using a direct socket.send call here because heartbeat requests are reserved by us.
|
||||
this.socket?.send(
|
||||
JSON.stringify({
|
||||
op: GatewayOpcodes.Heartbeat,
|
||||
d: this.previousSequenceNumber,
|
||||
}),
|
||||
)
|
||||
|
||||
this.heart.lastBeat = Date.now()
|
||||
this.heart.acknowledged = false
|
||||
|
||||
// After the random heartbeat jitter we can start a normal interval.
|
||||
this.heart.intervalId = setInterval(async () => {
|
||||
if (!this.isOpen()) return
|
||||
// gateway.debug("GW DEBUG", `Running setInterval in heartbeat file. Shard: ${shardId}`);
|
||||
|
||||
// gateway.debug("GW HEARTBEATING", { shardId, shard: currentShard });
|
||||
|
||||
// The Shard did not receive a heartbeat ACK from Discord in time,
|
||||
// therefore we have to assume that the connection has failed or got "zombied".
|
||||
// The Shard needs to start a re-identify action accordingly.
|
||||
// Reference: https://discord.com/developers/docs/topics/gateway#heartbeating-example-gateway-heartbeat-ack
|
||||
if (!this.heart.acknowledged) {
|
||||
this.close(ShardSocketCloseCodes.ZombiedConnection, 'Zombied connection, did not receive an heartbeat ACK in time.')
|
||||
|
||||
return await this.identify()
|
||||
}
|
||||
|
||||
this.heart.acknowledged = false
|
||||
|
||||
// Using a direct socket.send call here because heartbeat requests are reserved by us.
|
||||
this.socket?.send(
|
||||
JSON.stringify({
|
||||
op: GatewayOpcodes.Heartbeat,
|
||||
d: this.previousSequenceNumber,
|
||||
}),
|
||||
)
|
||||
|
||||
this.heart.lastBeat = Date.now()
|
||||
|
||||
this.events.heartbeat?.(this)
|
||||
}, this.heart.interval)
|
||||
}, jitter)
|
||||
}
|
||||
|
||||
/** Stop the heartbeating process with discord. */
|
||||
stopHeartbeating(): void {
|
||||
// Clear the regular heartbeat interval.
|
||||
clearInterval(this.heart.intervalId)
|
||||
// It's possible that the Shard got closed before the first jittered heartbeat.
|
||||
// To go safe we should clear the related timeout too.
|
||||
clearTimeout(this.heart.timeoutId)
|
||||
}
|
||||
|
||||
/**
|
||||
* Connects the bot user to a voice or stage channel.
|
||||
*
|
||||
* This function sends the _Update Voice State_ gateway command over the gateway behind the scenes.
|
||||
*
|
||||
* @param guildId - The ID of the guild the voice channel to leave is in.
|
||||
* @param channelId - The ID of the channel you want to join.
|
||||
*
|
||||
* @remarks
|
||||
* Requires the `CONNECT` permission.
|
||||
*
|
||||
* Fires a _Voice State Update_ gateway event.
|
||||
*
|
||||
* @see {@link https://discord.com/developers/docs/topics/gateway#update-voice-state}
|
||||
*/
|
||||
async joinVoiceChannel(
|
||||
guildId: BigString,
|
||||
channelId: BigString,
|
||||
options?: AtLeastOne<Omit<UpdateVoiceState, 'guildId' | 'channelId'>>,
|
||||
): Promise<void> {
|
||||
logger.debug(`[Shard] joinVoiceChannel guildId: ${guildId} channelId: ${channelId}`)
|
||||
return await this.send({
|
||||
op: GatewayOpcodes.VoiceStateUpdate,
|
||||
d: {
|
||||
guild_id: guildId.toString(),
|
||||
channel_id: channelId.toString(),
|
||||
self_mute: Boolean(options?.selfMute),
|
||||
self_deaf: options?.selfDeaf ?? true,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Edits the bot status in all shards that this gateway manages.
|
||||
*
|
||||
* @param data The status data to set the bots status to.
|
||||
* @returns Promise<void>
|
||||
*/
|
||||
async editBotStatus(data: StatusUpdate): Promise<void> {
|
||||
logger.debug(`[Shard] editBotStatus data: ${JSON.stringify(data)}`)
|
||||
return await this.editShardStatus(data)
|
||||
}
|
||||
|
||||
/**
|
||||
* Edits the bot's status on one shard.
|
||||
*
|
||||
* @param shardId The shard id to edit the status for.
|
||||
* @param data The status data to set the bots status to.
|
||||
* @returns Promise<void>
|
||||
*/
|
||||
async editShardStatus(data: StatusUpdate): Promise<void> {
|
||||
logger.debug(`[Shard] editShardStatus shardId: ${this.id} -> data: ${JSON.stringify(data)}`)
|
||||
return await this.send({
|
||||
op: GatewayOpcodes.PresenceUpdate,
|
||||
d: {
|
||||
since: null,
|
||||
afk: false,
|
||||
activities: data.activities,
|
||||
status: data.status,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches the list of members for a guild over the gateway.
|
||||
*
|
||||
* @param guildId - The ID of the guild to get the list of members for.
|
||||
* @param options - The parameters for the fetching of the members.
|
||||
*
|
||||
* @remarks
|
||||
* If requesting the entire member list:
|
||||
* - Requires the `GUILD_MEMBERS` intent.
|
||||
*
|
||||
* If requesting presences ({@link RequestGuildMembers.presences | presences} set to `true`):
|
||||
* - Requires the `GUILD_PRESENCES` intent.
|
||||
*
|
||||
* If requesting a prefix ({@link RequestGuildMembers.query | query} non-`undefined`):
|
||||
* - Returns a maximum of 100 members.
|
||||
*
|
||||
* If requesting a users by ID ({@link RequestGuildMembers.userIds | userIds} non-`undefined`):
|
||||
* - Returns a maximum of 100 members.
|
||||
*
|
||||
* Fires a _Guild Members Chunk_ gateway event for every 1000 members fetched.
|
||||
*
|
||||
* @see {@link https://discord.com/developers/docs/topics/gateway#request-guild-members}
|
||||
*/
|
||||
async requestMembers(guildId: BigString, options?: Omit<RequestGuildMembers, 'guildId'>): Promise<Camelize<DiscordMember[]>> {
|
||||
// You can request 1 member without the intent
|
||||
// Check if intents is not 0 as proxy ws won't set intents in other instances
|
||||
if (this.connection.intents && (!options?.limit || options.limit > 1) && !(this.connection.intents & GatewayIntents.GuildMembers)) {
|
||||
throw new Error('MISSING_INTENT_GUILD_MEMBERS')
|
||||
}
|
||||
|
||||
if (options?.userIds?.length) {
|
||||
logger.debug(`[Shard] requestMembers guildId: ${guildId} -> setting user limit based on userIds length: ${options.userIds.length}`)
|
||||
options.limit = options.userIds.length
|
||||
}
|
||||
|
||||
const nonce = `${guildId}-${Date.now()}`
|
||||
|
||||
// Gateway does not require caching these requests so directly send and return
|
||||
if (!this.cache.requestMembers?.enabled) {
|
||||
logger.debug(`[Shard] requestMembers guildId: ${guildId} -> skipping cache -> options ${JSON.stringify(options)}`)
|
||||
await this.send({
|
||||
op: GatewayOpcodes.RequestGuildMembers,
|
||||
d: {
|
||||
guild_id: guildId.toString(),
|
||||
// If a query is provided use it, OR if a limit is NOT provided use ""
|
||||
query: options?.query ?? (options?.limit ? undefined : ''),
|
||||
limit: options?.limit ?? 0,
|
||||
presences: options?.presences ?? false,
|
||||
user_ids: options?.userIds?.map((id) => id.toString()),
|
||||
nonce,
|
||||
},
|
||||
})
|
||||
return []
|
||||
}
|
||||
|
||||
return await new Promise((resolve) => {
|
||||
this.cache.requestMembers?.pending.set(nonce, { nonce, resolve, members: [] })
|
||||
|
||||
logger.debug(`[Shard] requestMembers guildId: ${guildId} -> requesting members -> data: ${JSON.stringify(options)}`)
|
||||
this.send({
|
||||
op: GatewayOpcodes.RequestGuildMembers,
|
||||
d: {
|
||||
guild_id: guildId.toString(),
|
||||
// If a query is provided use it, OR if a limit is NOT provided use ""
|
||||
query: options?.query ?? (options?.limit ? undefined : ''),
|
||||
limit: options?.limit ?? 0,
|
||||
presences: options?.presences ?? false,
|
||||
user_ids: options?.userIds?.map((id) => id.toString()),
|
||||
nonce,
|
||||
},
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Leaves the voice channel the bot user is currently in.
|
||||
*
|
||||
* This function sends the _Update Voice State_ gateway command over the gateway behind the scenes.
|
||||
*
|
||||
* @param guildId - The ID of the guild the voice channel to leave is in.
|
||||
*
|
||||
* @remarks
|
||||
* Fires a _Voice State Update_ gateway event.
|
||||
*
|
||||
* @see {@link https://discord.com/developers/docs/topics/gateway#update-voice-state}
|
||||
*/
|
||||
async leaveVoiceChannel(guildId: BigString): Promise<void> {
|
||||
logger.debug(`[Shard] leaveVoiceChannel guildId: ${guildId} Shard ${this.id}`)
|
||||
return await this.send({
|
||||
op: GatewayOpcodes.VoiceStateUpdate,
|
||||
d: {
|
||||
guild_id: guildId.toString(),
|
||||
channel_id: null,
|
||||
self_mute: false,
|
||||
self_deaf: false,
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
export interface ShardCreateOptions {
|
||||
/** The shard id */
|
||||
id: number
|
||||
/** The connection details */
|
||||
connection: ShardGatewayConfig
|
||||
/** The event handlers for events on the shard. */
|
||||
events: ShardEvents
|
||||
}
|
||||
|
||||
export default Shard
|
||||
@@ -0,0 +1,3 @@
|
||||
export * from './manager.js'
|
||||
export * from './Shard.js'
|
||||
export * from './types.js'
|
||||
@@ -0,0 +1,455 @@
|
||||
import type { AtLeastOne, BigString, Camelize, DiscordGetGatewayBot, DiscordMember, RequestGuildMembers } from '@discordeno/types'
|
||||
import type { LeakyBucket } from '@discordeno/utils'
|
||||
import { Collection, createLeakyBucket, delay, logger } from '@discordeno/utils'
|
||||
import Shard from './Shard.js'
|
||||
import type { ShardEvents, StatusUpdate, UpdateVoiceState } from './types.js'
|
||||
|
||||
export function createGatewayManager(options: CreateGatewayManagerOptions): GatewayManager {
|
||||
if (!options.connection) {
|
||||
options.connection = {
|
||||
url: 'wss://gateway.discord.gg',
|
||||
shards: 1,
|
||||
sessionStartLimit: {
|
||||
maxConcurrency: 1,
|
||||
remaining: 1000,
|
||||
total: 1000,
|
||||
resetAfter: 1000 * 60 * 60 * 24,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
const gateway: GatewayManager = {
|
||||
events: options.events,
|
||||
compress: options.compress ?? false,
|
||||
intents: options.intents ?? 0,
|
||||
properties: {
|
||||
os: options.properties?.os ?? process.platform,
|
||||
browser: options.properties?.browser ?? 'Discordeno',
|
||||
device: options.properties?.device ?? 'Discordeno',
|
||||
},
|
||||
token: options.token,
|
||||
url: options.url ?? options.connection.url ?? 'wss://gateway.discord.gg',
|
||||
version: options.version ?? 10,
|
||||
connection: options.connection,
|
||||
totalShards: options.totalShards ?? options.connection.shards ?? 1,
|
||||
lastShardId: options.lastShardId ?? 0,
|
||||
firstShardId: options.firstShardId ?? 0,
|
||||
totalWorkers: options.totalWorkers ?? 4,
|
||||
shardsPerWorker: options.shardsPerWorker ?? 25,
|
||||
spawnShardDelay: options.spawnShardDelay ?? 5300,
|
||||
shards: new Map(),
|
||||
buckets: new Map(),
|
||||
cache: {
|
||||
requestMembers: {
|
||||
enabled: options.cache?.requestMembers?.enabled ?? false,
|
||||
pending: new Collection(),
|
||||
},
|
||||
},
|
||||
|
||||
calculateTotalShards() {
|
||||
// Bots under 100k servers do not have access to total shards.
|
||||
if (gateway.totalShards < 100) {
|
||||
logger.debug(`[Gateway] Calculating total shards: ${gateway.totalShards}`)
|
||||
return gateway.totalShards
|
||||
}
|
||||
|
||||
logger.debug(`[Gateway] Calculating total shards`, gateway.totalShards, gateway.connection.sessionStartLimit.maxConcurrency)
|
||||
// Calculate a multiple of `maxConcurrency` which can be used to connect to the gateway.
|
||||
return (
|
||||
Math.ceil(
|
||||
gateway.totalShards /
|
||||
// If `maxConcurrency` is 1 we can safely use 16.
|
||||
(gateway.connection.sessionStartLimit.maxConcurrency === 1 ? 16 : gateway.connection.sessionStartLimit.maxConcurrency),
|
||||
) * gateway.connection.sessionStartLimit.maxConcurrency
|
||||
)
|
||||
},
|
||||
calculateWorkerId(shardId) {
|
||||
// Ignore decimal numbers.
|
||||
let workerId = Math.floor(shardId / gateway.shardsPerWorker)
|
||||
// If the workerId overflows the maximal allowed workers we by default just use to last worker.
|
||||
if (workerId >= gateway.totalWorkers) {
|
||||
// The Id of the last available worker is total -1
|
||||
workerId = gateway.totalWorkers - 1
|
||||
}
|
||||
|
||||
logger.debug(
|
||||
`[Gateway] Calculating workerId: Shard: ${shardId} -> Worker: ${workerId} -> Per Worker: ${gateway.shardsPerWorker} -> Total: ${gateway.totalWorkers}`,
|
||||
)
|
||||
|
||||
return workerId
|
||||
},
|
||||
prepareBuckets() {
|
||||
for (let i = 0; i < gateway.connection.sessionStartLimit.maxConcurrency; ++i) {
|
||||
logger.debug(`[Gateway] Preparing buckets for concurrency: ${i}`)
|
||||
gateway.buckets.set(i, {
|
||||
workers: [],
|
||||
leak: createLeakyBucket({
|
||||
max: 1,
|
||||
refillAmount: 1,
|
||||
// special number which is proven to be working dont change
|
||||
refillInterval: gateway.spawnShardDelay,
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
// ORGANIZE ALL SHARDS INTO THEIR OWN BUCKETS
|
||||
for (let shardId = gateway.firstShardId; shardId <= gateway.lastShardId; ++shardId) {
|
||||
logger.debug(`[Gateway] Preparing buckets for shard: ${shardId}`)
|
||||
if (shardId >= gateway.totalShards) {
|
||||
throw new Error(`Shard (id: ${shardId}) is bigger or equal to the used amount of used shards which is ${gateway.totalShards}`)
|
||||
}
|
||||
|
||||
const bucketId = shardId % gateway.connection.sessionStartLimit.maxConcurrency
|
||||
const bucket = gateway.buckets.get(bucketId)
|
||||
if (!bucket) {
|
||||
throw new Error(
|
||||
`Shard (id: ${shardId}) got assigned to an illegal bucket id: ${bucketId}, expected a bucket id between 0 and ${
|
||||
gateway.connection.sessionStartLimit.maxConcurrency - 1
|
||||
}`,
|
||||
)
|
||||
}
|
||||
|
||||
// FIND A QUEUE IN THIS BUCKET THAT HAS SPACE
|
||||
// const worker = bucket.workers.find((w) => w.queue.length < gateway.shardsPerWorker);
|
||||
const workerId = gateway.calculateWorkerId(shardId)
|
||||
const worker = bucket.workers.find((w) => w.id === workerId)
|
||||
if (worker) {
|
||||
// IF THE QUEUE HAS SPACE JUST ADD IT TO THIS QUEUE
|
||||
worker.queue.push(shardId)
|
||||
} else {
|
||||
bucket.workers.push({ id: workerId, queue: [shardId] })
|
||||
}
|
||||
}
|
||||
},
|
||||
async spawnShards() {
|
||||
// PREPARES ALL SHARDS IN SPECIFIC BUCKETS
|
||||
gateway.prepareBuckets()
|
||||
|
||||
// Prefer concurrency of forEach instead of forof
|
||||
await Promise.all(
|
||||
[...gateway.buckets.entries()].map(async ([bucketId, bucket]) => {
|
||||
for (const worker of bucket.workers) {
|
||||
for (const shardId of worker.queue) {
|
||||
await gateway.tellWorkerToIdentify(worker.id, shardId, bucketId)
|
||||
}
|
||||
}
|
||||
}),
|
||||
)
|
||||
},
|
||||
async shutdown(code, reason) {
|
||||
gateway.shards.forEach((shard) => shard.close(code, reason))
|
||||
|
||||
await delay(5000)
|
||||
},
|
||||
async tellWorkerToIdentify(workerId, shardId, bucketId) {
|
||||
logger.debug(`[Gateway] tell worker to identify (${workerId}, ${shardId}, ${bucketId})`)
|
||||
return await gateway.identify(shardId)
|
||||
},
|
||||
async identify(shardId: number) {
|
||||
let shard = this.shards.get(shardId)
|
||||
logger.debug(`[Gateway] identifying ${shard ? 'existing' : 'new'} shard (${shardId})`)
|
||||
|
||||
if (!shard) {
|
||||
shard = new Shard({
|
||||
id: shardId,
|
||||
connection: {
|
||||
compress: this.compress,
|
||||
intents: this.intents,
|
||||
properties: this.properties,
|
||||
token: this.token,
|
||||
totalShards: this.totalShards,
|
||||
url: this.url,
|
||||
version: this.version,
|
||||
},
|
||||
events: options.events,
|
||||
})
|
||||
|
||||
this.shards.set(shardId, shard)
|
||||
}
|
||||
|
||||
return await shard.identify()
|
||||
},
|
||||
async kill(shardId: number) {
|
||||
const shard = this.shards.get(shardId)
|
||||
if (!shard) {
|
||||
return logger.debug(`[Gateway] kill shard but not found (${shardId})`)
|
||||
}
|
||||
|
||||
logger.debug(`[Gateway] kill shard (${shardId})`)
|
||||
this.shards.delete(shardId)
|
||||
return await shard.shutdown()
|
||||
},
|
||||
|
||||
async requestIdentify() {
|
||||
logger.debug(`[Gateway] requesting identify`)
|
||||
},
|
||||
|
||||
// Helpers methods below this
|
||||
|
||||
calculateShardId(guildId, totalShards) {
|
||||
// If none is provided, use the total shards number from gateway object.
|
||||
if (!totalShards) totalShards = gateway.totalShards
|
||||
// If it is only 1 shard, it will always be shard id 0
|
||||
if (totalShards === 1) {
|
||||
logger.debug(`[Gateway] calculateShardId (1 shard)`)
|
||||
return 0
|
||||
}
|
||||
|
||||
logger.debug(`[Gateway] calculateShardId (guildId: ${guildId}, totalShards: ${totalShards})`)
|
||||
return Number((BigInt(guildId) >> 22n) % BigInt(totalShards))
|
||||
},
|
||||
|
||||
async joinVoiceChannel(guildId, channelId, options) {
|
||||
const shardId = gateway.calculateShardId(guildId)
|
||||
const shard = gateway.shards.get(shardId)
|
||||
if (!shard) {
|
||||
throw new Error(`Shard (id: ${shardId} not found`)
|
||||
}
|
||||
|
||||
logger.debug(`[Gateway] joinVoiceChannel guildId: ${guildId} channelId: ${channelId}`)
|
||||
shard.joinVoiceChannel(guildId, channelId, options)
|
||||
},
|
||||
|
||||
async editBotStatus(data) {
|
||||
logger.debug(`[Gateway] editBotStatus data: ${JSON.stringify(data)}`)
|
||||
await Promise.all(
|
||||
[...gateway.shards.values()].map(async (shard) => {
|
||||
gateway.editShardStatus(shard.id, data)
|
||||
}),
|
||||
)
|
||||
},
|
||||
|
||||
async editShardStatus(shardId, data) {
|
||||
const shard = gateway.shards.get(shardId)
|
||||
if (!shard) {
|
||||
throw new Error(`Shard (id: ${shardId}) not found.`)
|
||||
}
|
||||
|
||||
logger.debug(`[Gateway] editShardStatus shardId: ${shardId} -> data: ${JSON.stringify(data)}`)
|
||||
return await shard.editShardStatus(data)
|
||||
},
|
||||
|
||||
async requestMembers(guildId, options) {
|
||||
const shardId = gateway.calculateShardId(guildId)
|
||||
const shard = gateway.shards.get(shardId)
|
||||
if (!shard) {
|
||||
throw new Error(`Shard (id: ${shardId}) not found.`)
|
||||
}
|
||||
|
||||
logger.debug(`[Gateway] requestMembers guildId: ${guildId} -> options ${JSON.stringify(options)}`)
|
||||
return await shard.requestMembers(guildId, options)
|
||||
},
|
||||
|
||||
async leaveVoiceChannel(guildId) {
|
||||
const shardId = gateway.calculateShardId(guildId)
|
||||
const shard = gateway.shards.get(shardId)
|
||||
if (!shard) {
|
||||
throw new Error(`Shard (id: ${shardId} not found`)
|
||||
}
|
||||
|
||||
logger.debug(`[Gateway] leaveVoiceChannel guildId: ${guildId} Shard ${shardId}`)
|
||||
return await shard.leaveVoiceChannel(guildId)
|
||||
},
|
||||
}
|
||||
|
||||
return gateway
|
||||
}
|
||||
|
||||
export interface CreateGatewayManagerOptions {
|
||||
/**
|
||||
* Id of the first Shard which should get controlled by this manager.
|
||||
* @default 0
|
||||
*/
|
||||
firstShardId?: number
|
||||
/**
|
||||
* Id of the last Shard which should get controlled by this manager.
|
||||
* @default 0
|
||||
*/
|
||||
lastShardId?: number
|
||||
/**
|
||||
* Delay in milliseconds to wait before spawning next shard. OPTIMAL IS ABOVE 5100. YOU DON'T WANT TO HIT THE RATE LIMIT!!!
|
||||
* @default 5300
|
||||
*/
|
||||
spawnShardDelay?: number
|
||||
/**
|
||||
* Total amount of shards your bot uses. Useful for zero-downtime updates or resharding.
|
||||
* @default 1
|
||||
*/
|
||||
totalShards?: number
|
||||
/**
|
||||
* The amount of shards to load per worker.
|
||||
* @default 25
|
||||
*/
|
||||
shardsPerWorker?: number
|
||||
/**
|
||||
* The total amount of workers to use for your bot.
|
||||
* @default 4
|
||||
*/
|
||||
totalWorkers?: number
|
||||
/** Important data which is used by the manager to connect shards to the gateway. */
|
||||
connection?: Camelize<DiscordGetGatewayBot>
|
||||
/** Whether incoming payloads are compressed using zlib.
|
||||
*
|
||||
* @default false
|
||||
*/
|
||||
compress?: boolean
|
||||
/** The calculated intent value of the events which the shard should receive.
|
||||
*
|
||||
* @default 0
|
||||
*/
|
||||
intents?: number
|
||||
/** Identify properties to use */
|
||||
properties?: {
|
||||
/** Operating system the shard runs on.
|
||||
*
|
||||
* @default "darwin" | "linux" | "windows"
|
||||
*/
|
||||
os: string
|
||||
/** The "browser" where this shard is running on.
|
||||
*
|
||||
* @default "Discordeno"
|
||||
*/
|
||||
browser: string
|
||||
/** The device on which the shard is running.
|
||||
*
|
||||
* @default "Discordeno"
|
||||
*/
|
||||
device: string
|
||||
}
|
||||
/** Bot token which is used to connect to Discord */
|
||||
token: string
|
||||
/** The URL of the gateway which should be connected to.
|
||||
*
|
||||
* @default "wss://gateway.discord.gg"
|
||||
*/
|
||||
url?: string
|
||||
/** The gateway version which should be used.
|
||||
*
|
||||
* @default 10
|
||||
*/
|
||||
version?: number
|
||||
/** The events handlers */
|
||||
events: ShardEvents
|
||||
/** This managers cache related settings. */
|
||||
cache?: {
|
||||
requestMembers?: {
|
||||
/**
|
||||
* Whether or not request member requests should be cached.
|
||||
* @default false
|
||||
*/
|
||||
enabled?: boolean
|
||||
/** The pending requests. */
|
||||
pending: Collection<string, RequestMemberRequest>
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export interface GatewayManager extends Required<CreateGatewayManagerOptions> {
|
||||
/** The max concurrency buckets. Those will be created when the `spawnShards` (which calls `prepareBuckets` under the hood) function gets called. */
|
||||
buckets: Map<
|
||||
number,
|
||||
{
|
||||
workers: Array<{ id: number; queue: number[] }>
|
||||
leak: LeakyBucket
|
||||
}
|
||||
>
|
||||
/** The shards that are created. */
|
||||
shards: Map<number, Shard>
|
||||
/** Determine max number of shards to use based upon the max concurrency. */
|
||||
calculateTotalShards: () => number
|
||||
/** Determine the id of the worker which is handling a shard. */
|
||||
calculateWorkerId: (shardId: number) => number
|
||||
/** Prepares all the buckets that are available for identifying the shards. */
|
||||
prepareBuckets: () => void
|
||||
/** Start identifying all the shards. */
|
||||
spawnShards: () => Promise<void>
|
||||
/** Shutdown all shards. */
|
||||
shutdown: (code: number, reason: string) => Promise<void>
|
||||
/** Allows users to hook in and change to communicate to different workers across different servers or anything they like. For example using redis pubsub to talk to other servers. */
|
||||
tellWorkerToIdentify: (workerId: number, shardId: number, bucketId: number) => Promise<void>
|
||||
/** Tell the manager to identify a Shard. If this Shard is not already managed this will also add the Shard to the manager. */
|
||||
identify: (shardId: number) => Promise<void>
|
||||
/** Kill a shard. Close a shards connection to Discord's gateway (if any) and remove it from the manager. */
|
||||
kill: (shardId: number) => Promise<void>
|
||||
/** This function communicates with the parent manager, in order to know whether this manager is allowed to identify a new shard. */
|
||||
requestIdentify: () => Promise<void>
|
||||
/** Calculates the number of shards based on the guild id and total shards. */
|
||||
calculateShardId: (guildId: BigString, totalShards?: number) => number
|
||||
/**
|
||||
* Connects the bot user to a voice or stage channel.
|
||||
*
|
||||
* This function sends the _Update Voice State_ gateway command over the gateway behind the scenes.
|
||||
*
|
||||
* @param guildId - The ID of the guild the voice channel to leave is in.
|
||||
* @param channelId - The ID of the channel you want to join.
|
||||
*
|
||||
* @remarks
|
||||
* Requires the `CONNECT` permission.
|
||||
*
|
||||
* Fires a _Voice State Update_ gateway event.
|
||||
*
|
||||
* @see {@link https://discord.com/developers/docs/topics/gateway#update-voice-state}
|
||||
*/
|
||||
joinVoiceChannel: (guildId: BigString, channelId: BigString, options?: AtLeastOne<Omit<UpdateVoiceState, 'guildId' | 'channelId'>>) => Promise<void>
|
||||
/**
|
||||
* Edits the bot status in all shards that this gateway manages.
|
||||
*
|
||||
* @param data The status data to set the bots status to.
|
||||
* @returns Promise<void>
|
||||
*/
|
||||
editBotStatus: (data: StatusUpdate) => Promise<void>
|
||||
/**
|
||||
* Edits the bot's status on one shard.
|
||||
*
|
||||
* @param shardId The shard id to edit the status for.
|
||||
* @param data The status data to set the bots status to.
|
||||
* @returns Promise<void>
|
||||
*/
|
||||
editShardStatus: (shardId: number, data: StatusUpdate) => Promise<void>
|
||||
/**
|
||||
* Fetches the list of members for a guild over the gateway.
|
||||
*
|
||||
* @param guildId - The ID of the guild to get the list of members for.
|
||||
* @param options - The parameters for the fetching of the members.
|
||||
*
|
||||
* @remarks
|
||||
* If requesting the entire member list:
|
||||
* - Requires the `GUILD_MEMBERS` intent.
|
||||
*
|
||||
* If requesting presences ({@link RequestGuildMembers.presences | presences} set to `true`):
|
||||
* - Requires the `GUILD_PRESENCES` intent.
|
||||
*
|
||||
* If requesting a prefix ({@link RequestGuildMembers.query | query} non-`undefined`):
|
||||
* - Returns a maximum of 100 members.
|
||||
*
|
||||
* If requesting a users by ID ({@link RequestGuildMembers.userIds | userIds} non-`undefined`):
|
||||
* - Returns a maximum of 100 members.
|
||||
*
|
||||
* Fires a _Guild Members Chunk_ gateway event for every 1000 members fetched.
|
||||
*
|
||||
* @see {@link https://discord.com/developers/docs/topics/gateway#request-guild-members}
|
||||
*/
|
||||
requestMembers: (guildId: BigString, options?: Omit<RequestGuildMembers, 'guildId'>) => Promise<Camelize<DiscordMember[]>>
|
||||
/**
|
||||
* Leaves the voice channel the bot user is currently in.
|
||||
*
|
||||
* This function sends the _Update Voice State_ gateway command over the gateway behind the scenes.
|
||||
*
|
||||
* @param guildId - The ID of the guild the voice channel to leave is in.
|
||||
*
|
||||
* @remarks
|
||||
* Fires a _Voice State Update_ gateway event.
|
||||
*
|
||||
* @see {@link https://discord.com/developers/docs/topics/gateway#update-voice-state}
|
||||
*/
|
||||
leaveVoiceChannel: (guildId: BigString) => Promise<void>
|
||||
}
|
||||
|
||||
export interface RequestMemberRequest {
|
||||
/** The unique nonce for this request. */
|
||||
nonce: string
|
||||
/** The resolver handler to run when all members arrive. */
|
||||
resolve: (value: Camelize<DiscordMember[]> | PromiseLike<Camelize<DiscordMember[]>>) => void
|
||||
/** The members that have already arrived for this request. */
|
||||
members: Camelize<DiscordMember[]>
|
||||
}
|
||||
@@ -0,0 +1,181 @@
|
||||
import type { ActivityTypes, Camelize, DiscordActivity, DiscordGatewayPayload, GatewayOpcodes, PresenceStatus } from '@discordeno/types'
|
||||
import type Shard from './Shard.js'
|
||||
|
||||
export enum ShardState {
|
||||
/** Shard is fully connected to the gateway and receiving events from Discord. */
|
||||
Connected = 0,
|
||||
/** Shard started to connect to the gateway. This is only used if the shard is not currently trying to identify or resume. */
|
||||
Connecting = 1,
|
||||
/** Shard got disconnected and reconnection actions have been started. */
|
||||
Disconnected = 2,
|
||||
/** The shard is connected to the gateway but only heartbeating. At this state the shard has not been identified with discord. */
|
||||
Unidentified = 3,
|
||||
/** Shard is trying to identify with the gateway to create a new session. */
|
||||
Identifying = 4,
|
||||
/** Shard is trying to resume a session with the gateway. */
|
||||
Resuming = 5,
|
||||
/** Shard got shut down studied or due to a not (self) fixable error and may not attempt to reconnect on its own. */
|
||||
Offline = 6,
|
||||
}
|
||||
|
||||
export interface ShardGatewayConfig {
|
||||
/** Whether incoming payloads are compressed using zlib.
|
||||
*
|
||||
* @default false
|
||||
*/
|
||||
compress: boolean
|
||||
/** The calculated intent value of the events which the shard should receive.
|
||||
*
|
||||
* @default 0
|
||||
*/
|
||||
intents: number
|
||||
/** Identify properties to use */
|
||||
properties: {
|
||||
/** Operating system the shard runs on.
|
||||
*
|
||||
* @default "darwin" | "linux" | "windows"
|
||||
*/
|
||||
os: string
|
||||
/** The "browser" where this shard is running on.
|
||||
*
|
||||
* @default "Discordeno"
|
||||
*/
|
||||
browser: string
|
||||
/** The device on which the shard is running.
|
||||
*
|
||||
* @default "Discordeno"
|
||||
*/
|
||||
device: string
|
||||
}
|
||||
/** Bot token which is used to connect to Discord */
|
||||
token: string
|
||||
/** The URL of the gateway which should be connected to.
|
||||
*
|
||||
* @default "wss://gateway.discord.gg"
|
||||
*/
|
||||
url: string
|
||||
/** The gateway version which should be used.
|
||||
*
|
||||
* @default 10
|
||||
*/
|
||||
version: number
|
||||
/**
|
||||
* The total number of shards to connect to across the entire bot.
|
||||
* @default 1
|
||||
*/
|
||||
totalShards: number
|
||||
}
|
||||
|
||||
export interface ShardHeart {
|
||||
/** Whether or not the heartbeat was acknowledged by Discord in time. */
|
||||
acknowledged: boolean
|
||||
/** Interval between heartbeats requested by Discord. */
|
||||
interval: number
|
||||
/** Id of the interval, which is used for sending the heartbeats. */
|
||||
intervalId?: NodeJS.Timer
|
||||
/** Unix (in milliseconds) timestamp when the last heartbeat ACK was received from Discord. */
|
||||
lastAck?: number
|
||||
/** Unix timestamp (in milliseconds) when the last heartbeat was sent. */
|
||||
lastBeat?: number
|
||||
/** Round trip time (in milliseconds) from Shard to Discord and back.
|
||||
* Calculated using the heartbeat system.
|
||||
* Note: this value is undefined until the first heartbeat to Discord has happened.
|
||||
*/
|
||||
rtt?: number
|
||||
/** Id of the timeout which is used for sending the first heartbeat to Discord since it's "special". */
|
||||
timeoutId?: NodeJS.Timeout
|
||||
}
|
||||
|
||||
export interface ShardEvents {
|
||||
/** A heartbeat has been send. */
|
||||
heartbeat?: (shard: Shard) => unknown
|
||||
/** A heartbeat ACK was received. */
|
||||
heartbeatAck?: (shard: Shard) => unknown
|
||||
/** Shard has received a Hello payload. */
|
||||
hello?: (shard: Shard) => unknown
|
||||
/** The Shards session has been invalidated. */
|
||||
invalidSession?: (shard: Shard, resumable: boolean) => unknown
|
||||
/** The shard has started a resume action. */
|
||||
resuming?: (shard: Shard) => unknown
|
||||
/** The shard has successfully resumed an old session. */
|
||||
resumed?: (shard: Shard) => unknown
|
||||
/** Discord has requested the Shard to reconnect. */
|
||||
requestedReconnect?: (shard: Shard) => unknown
|
||||
/** The shard started to connect to Discord's gateway. */
|
||||
connecting?: (shard: Shard) => unknown
|
||||
/** The shard is connected with Discord's gateway. */
|
||||
connected?: (shard: Shard) => unknown
|
||||
/** The shard has been disconnected from Discord's gateway. */
|
||||
disconnected?: (shard: Shard) => unknown
|
||||
/** The shard has started to identify itself to Discord. */
|
||||
identifying?: (shard: Shard) => unknown
|
||||
/** The shard has successfully been identified itself with Discord. */
|
||||
identified?: (shard: Shard) => unknown
|
||||
/** The shard has received a message from Discord. */
|
||||
message?: (shard: Shard, payload: Camelize<DiscordGatewayPayload>) => unknown
|
||||
}
|
||||
|
||||
export enum ShardSocketCloseCodes {
|
||||
/** A regular Shard shutdown. */
|
||||
Shutdown = 3000,
|
||||
/** A resume has been requested and therefore the old connection needs to be closed. */
|
||||
ResumeClosingOldConnection = 3024,
|
||||
/** Did not receive a heartbeat ACK in time.
|
||||
* Closing the shard and creating a new session.
|
||||
*/
|
||||
ZombiedConnection = 3010,
|
||||
/** Discordeno's gateway tests hae been finished, therefore the Shard can be turned off. */
|
||||
TestingFinished = 3064,
|
||||
/** Special close code reserved for Discordeno's zero-downtime resharding system. */
|
||||
Resharded = 3065,
|
||||
/** Shard is re-identifying therefore the old connection needs to be closed. */
|
||||
ReIdentifying = 3066,
|
||||
}
|
||||
|
||||
export interface ShardSocketRequest {
|
||||
/** The OP-Code for the payload to send. */
|
||||
op: GatewayOpcodes
|
||||
/** Payload data. */
|
||||
d: unknown
|
||||
}
|
||||
|
||||
/** https://discord.com/developers/docs/topics/gateway-events#update-presence */
|
||||
export interface BotStatusUpdate {
|
||||
// /** Unix time (in milliseconds) of when the client went idle, or null if the client is not idle */
|
||||
since: number | null
|
||||
/** The user's activities */
|
||||
activities: BotActivity[]
|
||||
/** The user's new status */
|
||||
status: keyof typeof PresenceStatus
|
||||
}
|
||||
|
||||
/** https://discord.com/developers/docs/topics/gateway-events#activity-object */
|
||||
export interface BotActivity {
|
||||
name: string
|
||||
type: ActivityTypes
|
||||
url?: string
|
||||
}
|
||||
|
||||
/** https://discord.com/developers/docs/topics/gateway#update-voice-state */
|
||||
export interface UpdateVoiceState {
|
||||
/** id of the guild */
|
||||
guildId: string
|
||||
/** id of the voice channel client wants to join (null if disconnecting) */
|
||||
channelId: string | null
|
||||
/** Is the client muted */
|
||||
selfMute: boolean
|
||||
/** Is the client deafened */
|
||||
selfDeaf: boolean
|
||||
}
|
||||
|
||||
/** https://discord.com/developers/docs/topics/gateway-events#update-presence */
|
||||
export interface StatusUpdate {
|
||||
// /** Unix time (in milliseconds) of when the client went idle, or null if the client is not idle */
|
||||
// since: number | null;
|
||||
/** The user's activities */
|
||||
activities?: Camelize<Array<Omit<DiscordActivity, "created_at">>>
|
||||
/** The user's new status */
|
||||
status: keyof typeof PresenceStatus
|
||||
// /** Whether or not the client is afk */
|
||||
// afk: boolean;
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
import { Intents } from '@discordeno/types'
|
||||
import uWS from 'uWebSockets.js'
|
||||
import { createGatewayManager, ShardSocketCloseCodes } from '../../src/index.js'
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/explicit-function-return-type
|
||||
const createGatewayManagerWithPort = (port: number) =>
|
||||
createGatewayManager({
|
||||
connection: {
|
||||
url: `ws://localhost:${port}`,
|
||||
shards: 1,
|
||||
sessionStartLimit: {
|
||||
total: 1000,
|
||||
remaining: 998,
|
||||
resetAfter: 36579894,
|
||||
maxConcurrency: 1,
|
||||
},
|
||||
},
|
||||
token: ' ',
|
||||
url: `ws://localhost:${port}`,
|
||||
intents: Intents.Guilds,
|
||||
events: {},
|
||||
})
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/explicit-function-return-type
|
||||
const createUws = async (options: {
|
||||
onOpen?: () => any
|
||||
onMessage?: (message: any) => any
|
||||
onClose?: (code: number, message: string) => any
|
||||
closing?: boolean
|
||||
}) => {
|
||||
options.onOpen ??= () => {}
|
||||
options.onMessage ??= (message: any) => {}
|
||||
options.onClose ??= (code: number, message: string) => {}
|
||||
options.closing ??= false
|
||||
|
||||
return await new Promise<{ port: number; uwsToken: any }>((resolve, reject) => {
|
||||
let port = 0
|
||||
let uwsToken = 0
|
||||
uWS
|
||||
.App()
|
||||
.ws('/*', {
|
||||
compression: uWS.SHARED_COMPRESSOR,
|
||||
maxPayloadLength: 16 * 1024 * 1024,
|
||||
idleTimeout: 10,
|
||||
open: async (ws) => {
|
||||
if (options.closing) {
|
||||
ws.end(3000)
|
||||
return
|
||||
}
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
op: 10,
|
||||
d: {
|
||||
heartbeat_interval: 100,
|
||||
},
|
||||
}),
|
||||
)
|
||||
options.onOpen!()
|
||||
},
|
||||
message: async (ws, message, isBinary) => {
|
||||
const msg = JSON.parse(Buffer.from(message).toString())
|
||||
options.onMessage!(msg)
|
||||
if (msg.op === 1) {
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
op: 11,
|
||||
}),
|
||||
)
|
||||
return
|
||||
}
|
||||
if (msg.op === 2) {
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
t: 'READY',
|
||||
s: 1,
|
||||
op: 0,
|
||||
d: {
|
||||
v: 10,
|
||||
user_settings: {},
|
||||
user: {
|
||||
verified: true,
|
||||
username: 'testing bot',
|
||||
mfa_enabled: false,
|
||||
id: '000000707882254000',
|
||||
flags: 0,
|
||||
email: null,
|
||||
discriminator: '1687',
|
||||
bot: true,
|
||||
avatar: null,
|
||||
},
|
||||
shard: [0, 1],
|
||||
session_type: 'normal',
|
||||
session_id: '0dff79e1a6f2697388eb08924a0805c8',
|
||||
resume_gateway_url: `ws://localhost:${port}`,
|
||||
relationships: [],
|
||||
private_channels: [],
|
||||
presences: [],
|
||||
guilds: [],
|
||||
guild_join_requests: [],
|
||||
geo_ordered_rtc_regions: [],
|
||||
application: { id: '000000707882254000', flags: 27828224 },
|
||||
},
|
||||
}),
|
||||
)
|
||||
|
||||
return
|
||||
}
|
||||
if (msg.op === 6) {
|
||||
// resume
|
||||
}
|
||||
},
|
||||
close: (ws, code, message) => {
|
||||
const msg = Buffer.from(message).toString()
|
||||
options.onClose!(code, msg)
|
||||
},
|
||||
})
|
||||
.listen(0, async (token) => {
|
||||
if (!token) {
|
||||
reject(new Error())
|
||||
}
|
||||
// retrieve listening port
|
||||
uwsToken = token
|
||||
port = uWS.us_socket_local_port(token)
|
||||
resolve({ port, uwsToken })
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
describe('gateway', () => {
|
||||
it('can connect to server', async function () {
|
||||
this.timeout(6000)
|
||||
let resolveConnected: () => void
|
||||
const connected = new Promise<void>((resolve) => (resolveConnected = resolve))
|
||||
const uwsOptions = { onOpen: resolveConnected!, closing: false }
|
||||
const { port, uwsToken } = await createUws(uwsOptions)
|
||||
const gateway = createGatewayManagerWithPort(port)
|
||||
await gateway.spawnShards()
|
||||
await connected
|
||||
uwsOptions.closing = true
|
||||
await gateway.shutdown(ShardSocketCloseCodes.Shutdown, 'User requested bot stop')
|
||||
uWS.us_listen_socket_close(uwsToken)
|
||||
})
|
||||
|
||||
it('will heartbeat', async function () {
|
||||
this.timeout(6000)
|
||||
let resolveHeartbeat: () => void
|
||||
let resolveConnected: () => void
|
||||
const connected = new Promise<void>((resolve) => (resolveConnected = resolve))
|
||||
const Heartbeated = new Promise<void>((resolve) => (resolveHeartbeat = resolve))
|
||||
const uwsOptions = {
|
||||
onOpen: resolveConnected!,
|
||||
onMessage: (message: any) => {
|
||||
if (message.op !== 1) return
|
||||
resolveHeartbeat()
|
||||
},
|
||||
closing: false,
|
||||
}
|
||||
const { port, uwsToken } = await createUws(uwsOptions)
|
||||
const gateway = createGatewayManagerWithPort(port)
|
||||
await gateway.spawnShards()
|
||||
await connected
|
||||
const timeout = setTimeout(() => {
|
||||
throw new Error('Not heartbeat in time')
|
||||
}, 100)
|
||||
await Heartbeated
|
||||
clearTimeout(timeout)
|
||||
uwsOptions.closing = true
|
||||
await gateway.shutdown(ShardSocketCloseCodes.Shutdown, 'User requested bot stop')
|
||||
uWS.us_listen_socket_close(uwsToken)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1 @@
|
||||
export {}
|
||||
@@ -0,0 +1,7 @@
|
||||
import { describe, it } from 'mocha'
|
||||
|
||||
describe('index.ts', () => {
|
||||
it('will import without error', async () => {
|
||||
await import('../../src/index.js')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"extends": "tsconfig/base.json",
|
||||
"compilerOptions": {
|
||||
"outDir": "./dist",
|
||||
},
|
||||
"include": [
|
||||
"./src/**/*.ts",
|
||||
"./src/**/*.tsx"
|
||||
],
|
||||
"exclude": [
|
||||
"node_modules",
|
||||
"dist",
|
||||
"test",
|
||||
"tests"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"extends": "tsconfig/test.json",
|
||||
"include": [
|
||||
"tests",
|
||||
],
|
||||
"exclude": [
|
||||
"node_modules",
|
||||
"dist",
|
||||
"src"
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user