API - feature requests

  • Hi,

    I'm a heavy user of the new API and therefore I would like to propose some new features and some changes to make the API more consistent.

    Changes:

    • Summarize all user related stuff among /user
      • /user/[UserName]
      • /user/[UserName]/stats/[datetime] (currently /userstats/[UserName]/[datetime])
    • Split up key (date) and value (online time) of /userstats
    • Remove the 'writeProtectedSet' key of /land/[UserName]/[world]
    • Change the structure of the object of /shop/[UserName]/[world] to an array of that kind of object:
      • { "wallet": <number: accountId>, "shops": <array: shopdata> }:thumbup:
    • Change the structure of the object of /server/playerlocations/[world] to an array of that kind of object:
      • { "user": <string: uuid>, "pos": <location obj>}

    New Features:

    • Add an API-endpoint to query UserNames from UUIDs and vice versa for multiple UserNames/UUiDs at once (this is required for /land, /shop, /transaction, /user). :thumbup:
    • Add an API-endpoint to gather all transactions of a week/month :thumbup:

    :thumbup: = DONE

    Edited 4 times, last by d4n1el89 (July 10, 2020 at 2:42 PM).

  • /push

    To show the full functionality of the API a single change is needed yet:

    Change the structure of the object of /shop/[UserName]/[world] to an array of that kind of object:

    • { "wallet": <number: accountId>, "shops": <array: shopdata> }:thumbup:

    Edited once, last by d4n1el89 (July 10, 2020 at 2:42 PM).

  • hi,

    if possible, it would be great to get more generic information from the Api

    like serverstatus, number of player and maybe their names.

    and do you plan to add the progress in research/skill to the callable userdata?

  • Hey,

    as already talked about on the stream, I would love if you could get all transactions that were executed after another transaction. This would allow clients to get all unknown transcations without having to go through all days/months manually. Also, if I was to check multiple times a day if a new transacation took place, I wouldn't have to request the entire day all the time, but could instead call "after" with the latest known transaction.

    When first discussed on the stream, you meant that you would probably do this offstream, since this would require you to restructure the database for you to be able to query this efficiently.

    Example:

    Code
    https://s200.etm.dev/transaction/[UserName]/after/[TransactionId]

    called as https://s200.etm.dev/transaction/125m125/after/GVDLYD0P could return

    Bonus points, if the API supported long-polling requests, which don't return an empty array but instead block until a new transaction arrives, if the requested one is currently the latest one. This should then probably be configurable with a query parameter, for example https://s200.etm.dev/transaction/125m125/after/GVDLYD0P?longpolling=true

  • get all transactions that were executed after another transaction.

    Oh i remember now why i did not yet do this. The way i store the data in TransactionIDS is not ordered in the DB. So to actually get all transaction past a specific one i would need to check ALL transactions currently.
    Before implementing this i wanted to change this that the file/db representation is a normal incrementing ID and not the actual TransactionID visible to clients.


    supported long-polling requests,

    Probably not. If i go to that length it will most likely be websockets. Afaik they win the pros/cons fight over long polling (and i've done that already)

  • Probably not. If i go to that length it will most likely be websockets. Afaik they win the pros/cons fight over long polling (and i've done that already)

    I agree that Websockets are most likely a lot better, especially if you are able to watch multiple "topics" for changes. Just thought I could try my luck, since long polling requests are a lot easier to implement for clients that don't really have to worry about performance/open connections/... :saint:.

    If you are planning to use the websocket only for sending updates from the server to the client, server sent events could potentially also be interesting, although I have to admit, that I have never used them before.

  • server sent events could potentially also be interesting

    You are just talking about getting websocket messages if something happens on the server right - that would be the way to inform you similar to long-pulling.

    It will look something like this:

    Code
    {
      "subject": "xxxxxxxxx",
      "data": {}
    }

    Where data is any kind of json object and the subject would be a unique identifier for the data object

    Interpretation than just would be something like (typescript/angular)

  • Yes, overall it wouldn't be that much more complicated.

    With Websockets you just have a lot more "management" to do (depending on your library), like handling authentication (browser websockets don't support headers, if I remember correctly. So you would, for example, have to send your credentials as a message and handle the response (success vs failure)), have to tell the server which events you are interested in (and handle the response, if you are not automatically subscribed to all), have to handle disconnections (reestablish connection and after it is established do the "normal" http request to make sure you didn't miss anything).

    But I will most likely adapt to the technology you are using ;)

    So the code could look similar to this (and now that I have kind of written it anyway, I would be fine with Websockets ;) ). Line 59-70 in a loop would be all you need for long polling as a client.

  • Something like this should now work:

    Code
    https://s200.etm.dev/transaction/Kademlia/after/0Y1QW1N2
    
    path = "/{username:.+}/after/{transactionId:.+}"

    Seems to be working :love:. Is there a limit on the number of transactions returned?

    The timestamp attribute of the transaction itself seems to be broken. It returns 0 for all of them (not only for endpoint after, but also for latest and day/month) :

  • We are missing the permission kadi.usercommand.setsecret to be able to set the password for the new API/Websockets with the command /user setsecret <password>.

    But by stealing the one time password from the WebsocketTest-Link, I was able to get the login, ws-connection and Desktop-requests working. I assume the Transaction-APIs etc are not migrated to the new api (401 with old login method and 404 with x-auth-token)?

    Not really an error, but the s200.etm.dev-Frontend uses the hardcoded displayName "Kademlia" from the login-response as the name in the top right and for the first get-request of the desktop image, but that should solve itself when the login-request (or profile-/... in the future) returns the correct username. Subsequent updates from the websocket are working.