Broken with minecraft-protocol 0.16, missing protocol property #19

deathcap · 2016-01-02T00:52:44Z

Using the latest releases minecraft-protocol 0.16.5 and mineflayer 1.5.3, wsmc fails with:

node wsmc.js 
wsmc/wsmc.js:4
var readVarInt = minecraft_protocol.protocol.types.varint[0];
                                            ^

TypeError: Cannot read property 'types' of undefined
    at Object.<anonymous> (wsmc/wsmc.js:4:45)
    at Module._compile (module.js:398:26)
    at Object.Module._extensions..js (module.js:405:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Function.Module.runMain (module.js:430:10)
    at startup (node.js:141:18)
    at node.js:980:3

The text was updated successfully, but these errors were encountered:

deathcap · 2016-01-02T00:56:47Z

Works with minecraft-protocol@0.13.4, fails with minecraft-protocol@0.14.0

bisected to: PrismarineJS/node-minecraft-protocol@1a9e08c move createPacketBuffer and parsePacketData functions to serializer, also move protocol's exports to serializer

Looks like minecraft_protocol.protocol. changed to just minecraft_protocol. Works up to 0.15.0, but then in the next release 0.16.0, changed again:

wsmc/wsmc.js:4
var readVarInt = minecraft_protocol.types.varint[0];
                                         ^

TypeError: Cannot read property 'varint' of undefined
    at Object.<anonymous> (wsmc/wsmc.js:4:42)
    at Module._compile (module.js:398:26)
    at Object.Module._extensions..js (module.js:405:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Function.Module.runMain (module.js:430:10)
    at startup (node.js:141:18)
    at node.js:980:3

…le API). GH-19

deathcap · 2016-01-02T01:27:52Z

Began to update to 0.15.0 on https://github.com/deathcap/wsmc/compare/nmp0.15.0 - but node examples/mcwebchat/mcwebchat.js fails with this mysterious error:

wsmc/node_modules/prismarine-block/index.js:5
  var mcData=require('minecraft-data')(mcVersion);
                                      ^

TypeError: require(...) is not a function
    at loader (wsmc/node_modules/prismarine-block/index.js:5:39)
    at Object.<anonymous> (wsmc/node_modules/mineflayer/lib/plugins/blocks.js:5:40)
    at Module._compile (module.js:398:26)
    at Object.Module._extensions..js (module.js:405:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Module.require (module.js:354:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (wsmc/mineflayer-stream.js:11:15)
    at Module._compile (module.js:398:26)

Probably ought to just update straight to minecraft-protocol 0.16.0. But what version specifier should I use for API compatibility with this module? ^0.15.0 would pickup ^0.16.0. since "^" only locks the major version, accepts changes minor and patch - it may be acceptable to use ~0.15.0, to lock major and minor, only allowing updates from the patch version number. Or stricter, '0.15.0' (or 0.16.5, etc.) to lock to an exact version.

Ref #19

deathcap · 2016-01-02T03:22:36Z

Caused by ancient minecraft-data (0.0.1), updated to ~0.16.3, fixed above 'is not a function' error with minecraft-data but now hitting an incompatibility with mineflayer (1.5.3):

$ node examples/mcwebchat/mcwebchat.js 
You are using a pure-javascript implementation of RSA.
Your performance might be subpar. Please consider installing URSA
module.js:328
    throw err;
    ^

Error: Cannot find module 'mineflayer/lib/block'
    at Function.Module._resolveFilename (module.js:326:15)
    at Function.Module._load (module.js:277:25)
    at Module.require (module.js:354:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (wsmc/mineflayer-stream.js:41:10)
    at Module._compile (module.js:398:26)
    at Object.Module._extensions..js (module.js:405:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Module.require (module.js:354:17)

Block and Biome moved to prismarine-block and prismarine-biome in PrismarineJS/mineflayer@ca365c3

update: Block and Biome are still accessible via the mineflayer exports; fixed in d941812

mcwebchat now errors out in the browser:

Error: Cannot find module './minecraft-data/1.8/enums/blocks'
s_prelude.js:1
(anonymous function)_prelude.js:1
mcVersionToMcDataindex.js:30
exportsindex.js:9
(anonymous function)version.js:3
s_prelude.js:1
(anonymous function)_prelude.js:1
(anonymous function)browser.js:2

dynamic require in ./node_modules/minecraft-protocol/node_modules/minecraft-data/index.js:

        function mcVersionToMcData(mcVersion)
        {
            var dir = "./minecraft-data/" + mcVersion;
            return {
                blocks: require(dir + '/enums/blocks'),

static requires were added in PrismarineJS/node-minecraft-data@eb92ebd as of minecraft-data 0.15.0 (right after 0.14.0). minecraft-protocol 0.15.0 uses minecraft-data 0.11.0, so probably have to update straight to minecraft-protocol 0.16.

…iome mineflayer's lib/block.js and lib/biome.js moved to separate modules, prismarine-block and prismarine-biome. Use them indirectly through mineflayer's export object instead of referencing the now-nonexistent library files. GH-19

deathcap · 2016-01-09T06:49:29Z

Starting update to minecraft-protocol 0.16.6, failure on nmp0.15.0 branch is:

wsmc/wsmc.js:4
var readVarInt = minecraft_protocol.types.varint[0];
                                         ^

TypeError: Cannot read property 'varint' of undefined
    at Object.<anonymous> (wsmc/wsmc.js:4:42)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:313:12)
    at Function.Module.runMain (module.js:467:10)
    at startup (node.js:136:18)
    at node.js:963:3

major change is move to use protodef, relevant commit: PrismarineJS/node-minecraft-protocol@f45c6df

packet IDs also moved:

wsmc.js:24
var ids = minecraft_protocol.packetIds.play.toClient;
                                      ^

TypeError: Cannot read property 'play' of undefined
    at Object.<anonymous> (wsmc/wsmc.js:24:39)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:313:12)
    at Function.Module.runMain (module.js:467:10)
    at startup (node.js:136:18)
    at node.js:963:3

in PrismarineJS/node-minecraft-protocol@1a9e08c#diff-1fdf421c05c1140f6d71444ea2b27638L4

then to minecraft-data: PrismarineJS/node-minecraft-protocol@6b6303b#diff-1fdf421c05c1140f6d71444ea2b27638L7

Reference: PrismarineJS/node-minecraft-protocol@f45c6df ref #19

#19

deathcap · 2016-01-09T07:25:45Z

wsmc/JavaScript server now runs, but the mcwebchat example (browserified) fails in a dynamic require:

        function mcVersionToMcData(mcVersion) // mcVersion = '1.8'
        {
            var dir = "./minecraft-data/" + mcVersion;
            return {
                blocks: require(dir + '/enums/blocks'),

since browserify compiles requires statically, unable to resolve at runtime:

require('./minecraft-data/1.8/enums/blocks')
Error: Cannot find module './minecraft-data/1.8/enums/blocks'

fixed in node-minecraft-data 0.15.0: PrismarineJS/node-minecraft-data@eb92ebd - but mineflayer was using ~0.5.0. PR to mineflayer to update: PrismarineJS/mineflayer#354

gets further, until mineflayer/index.js using requireindex:

var requireIndex = require('requireindex');
var plugins = requireIndex(path.join(__dirname, 'lib', 'plugins'));

requireindex calls FS.readdirSync - not defined in browserify: (I thought it used to be? can be static)

                var files = FS.readdirSync(dir);

update: https://www.npmjs.com/package/bulkify browserify transform

brfs transform supports fs.readdirSync: browserify/brfs#19 (PR merged)

For supporting requireindex in mineflayer, see porting progress #19

deathcap · 2016-01-09T20:00:12Z

Added brfs transform but now running into https://github.com/rom1504/node-mojangson browserify incompatibility:

 mcwebchat@0.0.1 start wsmc/examples/mcwebchat
> wzrd mcwebchat.js:bundle.js -- --global-transform brfs

server started at http://localhost:9966
{"url":"/bundle.js","type":"bundle","command":"browserify mcwebchat.js --global-transform brfs","elapsed":"1609ms","time":"2016-01-09T19:59:38.656Z"}
TypeError: callee.apply is not a function while parsing file: mineflayer/node_modules/node-mojangson/grammar.js
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:89:27)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:92:23)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:77:26)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:85:25)
    at module.exports (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:114:7)
    at traverse (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/index.js:256:23)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/index.js:208:13)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/falafel/index.js:49:9)
    at wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/falafel/index.js:46:17
    at forEach (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/foreach/index.js:12:16)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/falafel/index.js:34:9)

unable to repro in isolation (wzrd example.js in node-mojangson, the example parses), maybe an unintended interaction with global brfs browser transform or something else. repro with: browserify mcwebchat.js -g brfs

https://github.com/substack/static-eval/blob/c702b3e00091e0d351303796ff41c991e7babe62/index.js#L89 - static-eval walking CallExpression

       else if (node.type === 'CallExpression') {
            var callee = walk(node.callee);
            if (callee === FAIL) return FAIL;

            var ctx = node.callee.object ? walk(node.callee.object) : FAIL;
            if (ctx === FAIL) ctx = null;

            var args = [];
            for (var i = 0, l = node.arguments.length; i < l; i++) {
                var x = walk(node.arguments[i]);
                if (x === FAIL) return FAIL;
                args.push(x);
            }
            return callee.apply(ctx, args);
        }

the callee is an object: { resolve: [Function: resolver] }, args are [ "path" ]

ah, node-mojangson has a dynamic readFileSync in the generated grammar.js:

var source = require('fs').readFileSync(require('path').normalize(args[1]), "utf8");

but this shouldn't crash. Proposed a fix in static-eval (-> static-module -> static-module -> brfs):

browserify/static-eval#12 Fix exception instead of failure when parsing CallExpressions

and removing the affected (unused) unbrowserifiable code in node-mojangson:

PrismarineJS/node-mojangson#4

deathcap · 2016-01-10T03:41:07Z

mojangson 0.2.2 is in mineflayer 2570e20, testing with it able to brfs global transform now with no parse error - but the transformed requireindex crashes here:

          var files = FS.readdirSync(dir);

with no defined 'FS' (?) - this likely cannot be statically replaced. But mineflayer has had:

        var plugins = requireIndex(path.join(__dirname, 'lib', 'plugins'));

for a while, since at least PrismarineJS/mineflayer@bd4eba1 - not sure how I had this working in browserify previously

deathcap · 2016-01-12T04:52:14Z

Mineflayer plugins were statically require()'d in wsmc/mineflayer-stream.js:

wsmc/mineflayer-stream.js

Lines 8 to 31 in 57ceb6a

    
           , plugins = { 
        
               bed: require('mineflayer/lib/plugins/bed'), 
        
               block_actions: require('mineflayer/lib/plugins/block_actions'), 
        
               blocks: require('mineflayer/lib/plugins/blocks'), 
        
               chat: require('mineflayer/lib/plugins/chat'), 
        
               chest: require('mineflayer/lib/plugins/chest'), 
        
               craft: require('mineflayer/lib/plugins/craft'), 
        
               creative: require('mineflayer/lib/plugins/creative'), 
        
               digging: require('mineflayer/lib/plugins/digging'), 
        
               dispenser: require('mineflayer/lib/plugins/dispenser'), 
        
               enchantment_table: require('mineflayer/lib/plugins/enchantment_table'), 
        
               entities: require('mineflayer/lib/plugins/entities'), 
        
               experience: require('mineflayer/lib/plugins/experience'), 
        
               game: require('mineflayer/lib/plugins/game'), 
        
               health: require('mineflayer/lib/plugins/health'), 
        
               inventory: require('mineflayer/lib/plugins/inventory'), 
        
               kick: require('mineflayer/lib/plugins/kick'), 
        
               physics: require('mineflayer/lib/plugins/physics'), 
        
               rain: require('mineflayer/lib/plugins/rain'), 
        
               settings: require('mineflayer/lib/plugins/settings'), 
        
               simple_inventory: require('mineflayer/lib/plugins/simple_inventory'), 
        
               sound: require('mineflayer/lib/plugins/sound'), 
        
               spawn_point: require('mineflayer/lib/plugins/spawn_point'), 
        
               time: require('mineflayer/lib/plugins/time')

- this plugin list (via the dynamic requireIndex) wasn't supposed to be used, but require('mineflayer') is called from the change to use require('mineflayer').Block (etc.) instead of require('mineflayer/lib/block') I had to in order to get prismarine-block: d941812

This puts browserified mineflayer in a bit of a pickle:

I can't change back to require('mineflayer/lib/block') because block.js no longer exists in mineflayer (and neither does biome.js): https://github.com/PrismarineJS/mineflayer/tree/master/lib - they were moved to separate modules: PrismarineJS/mineflayer@ca365c3 - prismarine-block and prismarine-biome.
If I change to require('mineflayer').Block to require("prismarine-block")(mc.minecraftVersion) (and likewise for biome), I'll have to include prismarine-block in my project, and ensure it matches the same version as mineflayer uses. If other objects move to their own modules, then I'll have to update my project to the new modules accordingly, increasing maintenance difficulty and version fragility.
If I use require('mineflayer').Block then mineflayer/index.js will execute and hit the unbrowserifiable requireindex (maybe it only worked before due Doesn't handle var statements with multiple declarations browserify/static-eval#10 Doesn't handle var statements with multiple declarations?)

I think any solution will require changes to mineflayer, but I'm not sure what is the correct fix going forward. Some ideas:

Restore mineflayer/lib/block.js and biome, but have them just require prismarine-block, restoring require('mineflayer/lib/block') compatibility
Shim requireindex (sidenode: this is https://www.npmjs.com/package/requireindex not https://www.npmjs.com/package/require-index) for browserify so it does nothing (since the plugins list is mineflayer's index.js is not used here)
Change mineflayer/index.js to not use requireindex and just list the plugins statically like I do in wsmc/mineflayer-stream.js. Requires some more maintenance in mineflayer, as new plugins are added, but makes it directly compatible with browserify. Straightforward easy change.

update: above mineflayer change works, PR'd for consideration: PrismarineJS/mineflayer#356

Add a mineflayer browser.js alternative replacement for index.js with the static requires. Basically https://github.com/deathcap/wsmc/blob/d94181283b588b127269d1aed90d5d40d9964915/mineflayer-stream.js instead of https://github.com/PrismarineJS/mineflayer/blob/c3f67f98c7002c8aad720fde9ad280bde85dd4ac/index.js - but another change I have is using WS instead of TCP. There is some interesting in WS support in node-minecraft-protocol: WebSocket support PrismarineJS/node-minecraft-protocol#312 - this could be a part of it. Would make wsmc simpler and easier to maintain if all the WS stuff was in NMP.
Something else?

deathcap · 2016-01-12T05:43:40Z

After solving the above issue, next step is to update wsmc/minecraft-protocol-stream.js for the changes in node-minecraft-protocol/createClient.js.

#19 (comment)

deathcap · 2016-01-12T06:03:07Z

Current status: wsmc runs, but on ws connection, logs a bunch of errors (probably not transitioning states correctly — the ws is supposed to mainly be in the PLAY state):

Successfully connected to MC
compress { threshold: 256 }
Skipping state [object Object] packet:  <Buffer 03 80 02>
Skipping state [object Object] packet:  <Buffer 02 24 34 34 63 63 61 36 61 34 2d 34 30 36 65 2d 33 38 36 36 2d 39 35 63 64 2d 64 34 63 65 38 33 64 62 38 34 63 61 09 77 65 62 75 73 65 72 2d 36>
…

no chat messages received, attempting to send a message in mcwebchat fails with:

WebSocket error: Error: Serialization error for .toServer : SizeOf error for name : chat is not in the mappings value

stack trace points to ProtoDef but doesn't say where it is called:

rom1504 · 2016-01-12T10:49:13Z

"chat is not in the mappings value" means it's not in the correct state indeed. That error definitely doesn't say enough though.

rom1504 · 2016-01-12T10:54:29Z

I think that's happening somewhere you do a .write('chat',...) and you are not in PLAY state.

deathcap · 2016-01-13T06:28:25Z

It's throwing the exception here:

mappings is only {0: "login_start", 1: "encryption_begin"}, so it's definitely in the wrong state.

Receiving the connect event, changes client.state in onConnect() to LOGIN, but not getting the success event which calls onLogin() to set client.state to PLAY. wsmc is logging:

    if (state !== 'play' && state !== 'login') {
      console.log('Skipping state '+state+' packet: ',buffer);
      return;
    }

state is now an object { size: 6, name: 'keep_alive', state: 'play' } - need to update this check.

#19 (comment)

deathcap · 2016-01-13T07:12:57Z

Tracking further updates in the pull request: #21

deathcap · 2016-01-17T04:18:40Z

Fixed in #21 c44de84

deathcap added a commit that referenced this issue Jan 2, 2016

Start updating to minecraft-protocol ~0.15.0 (not ^0.16.0; incompatib…

d51effc

…le API). GH-19

deathcap added a commit that referenced this issue Jan 2, 2016

Update to minecraft-data ~0.16.3

a2eb761

Ref #19

This was referenced Jan 9, 2016

Implement a prismarine-world web visualizer PrismarineJS/prismarine-world#17

Closed

WebSocket support PrismarineJS/node-minecraft-protocol#312

Open

deathcap added a commit that referenced this issue Jan 9, 2016

varint parsing moved to protodef

3743900

Reference: PrismarineJS/node-minecraft-protocol@f45c6df ref #19

deathcap added a commit that referenced this issue Jan 9, 2016

Packets moved to minecraft-data

8466191

#19

deathcap added a commit that referenced this issue Jan 9, 2016

Add global brfs browserify transform to mcwebchat example

deb342e

For supporting requireindex in mineflayer, see porting progress #19

deathcap mentioned this issue Jan 12, 2016

Remove requireindex, for browserify support PrismarineJS/mineflayer#356

Merged

deathcap added a commit that referenced this issue Jan 12, 2016

State move to minecraft-data

2fbfd59

#19 (comment)

deathcap mentioned this issue Jan 12, 2016

Update to node-minecraft-protocol 0.16 (major changes including using protodef) #21

Merged

deathcap added a commit that referenced this issue Jan 13, 2016

Fix wsmc protocol state check in raw handler, now an object

d7f2305

#19 (comment)

deathcap closed this as completed Jan 17, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Broken with minecraft-protocol 0.16, missing protocol property #19

Broken with minecraft-protocol 0.16, missing protocol property #19

deathcap commented Jan 2, 2016

deathcap commented Jan 2, 2016

deathcap commented Jan 2, 2016

deathcap commented Jan 2, 2016

deathcap commented Jan 9, 2016

deathcap commented Jan 9, 2016

deathcap commented Jan 9, 2016

deathcap commented Jan 10, 2016

deathcap commented Jan 12, 2016

deathcap commented Jan 12, 2016

deathcap commented Jan 12, 2016

rom1504 commented Jan 12, 2016

rom1504 commented Jan 12, 2016

deathcap commented Jan 13, 2016

deathcap commented Jan 13, 2016

deathcap commented Jan 17, 2016

Broken with minecraft-protocol 0.16, missing protocol property #19

Broken with minecraft-protocol 0.16, missing protocol property #19

Comments

deathcap commented Jan 2, 2016

deathcap commented Jan 2, 2016

deathcap commented Jan 2, 2016

deathcap commented Jan 2, 2016

deathcap commented Jan 9, 2016

deathcap commented Jan 9, 2016

deathcap commented Jan 9, 2016

deathcap commented Jan 10, 2016

deathcap commented Jan 12, 2016

deathcap commented Jan 12, 2016

deathcap commented Jan 12, 2016

rom1504 commented Jan 12, 2016

rom1504 commented Jan 12, 2016

deathcap commented Jan 13, 2016

deathcap commented Jan 13, 2016

deathcap commented Jan 17, 2016