dwalton76
Published © MIT

CraneCuber

CraneCuber is a voice controlled robot that solves 2x2x2 up to 6x6x6 Rubik's Cubes! It is a combo of a LEGO crane, EV3 Mindstorms and Alexa.

AdvancedFull instructions provided8 hours460
CraneCuber

Things used in this project

Hardware components

EV3 Programming Brick / Kit
LEGO MindStorms EV3 Programming Brick / Kit
×1
Echo Dot
Amazon Alexa Echo Dot
×1
Webcam, Logitech® HD Pro
Webcam, Logitech® HD Pro
Any webcam will do
×1
LEGO MindStorms LEGO Technic 42009 Mobile Crane MK II
This set is no longer produced but the user could easily purchase the needed parts from www.bricklink.com. Not all parts from this set were used in CraneCuber.
×1
5x5x5 Rubiks Cube
Any rubiks cube from a 2x2x2 up to 6x6x6 will work.
×1
USB Expansion HUB, USB 2.0 Powered
USB Expansion HUB, USB 2.0 Powered
×1
usb wifi dongle
×1

Software apps and online services

rubiks-cube-tracker
This is software that I wrote, it is open source and is available on github. rubiks-cube-tracker analyzes a photograph or video feed of a rubiks cube and locates the cube in the image. It then locates all of the "squares" of the cube in the image and extract the RGB (red, green, blue) values for each square. Here is a youtube video of this software in action: https://www.youtube.com/watch?v=3tWnl9rLnfE And a blog article I wrote about how it works: http://programmablebrick.blogspot.com/2017/02/rubiks-cube-tracker-using-opencv.html
rubiks-color-resolver
This is software that I wrote, it is open source and is available on github. rubiks-color-resolver accepts a list of the RGB (red, green, blue) values for all squares on the cube and determines the exact color (white, yellow, red, orange, blue or green) of each square. This determines the exact state of the cube which will be needed by the solver to compute a solution. I wrote a blog article about how it works, just scroll down to the "Color Extraction" section http://programmablebrick.blogspot.com/2017/02/rubiks-cube-tracker-using-opencv.html
rubiks-cube-NxNxN-solver
This is software that I wrote, it is open source and is available on github. To my knowledge rubiks-cube-NxNxN-solver is the only open source NxNxN solver in the world. It is capable of solving any size rubiks cube, I have tested up to a 17x17x17!! It was about 8 months of work to progress from not knowing how to solve a rubiks cube at all to being able to solve NxNxN.
lego-crane-cuber
This is software that I wrote, it is open source and is available on github. This is the CraneCuber program that runs on the EV3.

Story

Read more

Schematics

Build Instructions

I created these using https://www.bricklink.com/v2/build/studio.page

Parts List

Code

skill.json

JSON
The JSON format for the "Build" section of my Alexa CraneCuber skill
{
    "interactionModel": {
        "languageModel": {
            "invocationName": "crane cuber",
            "intents": [
                {
                    "name": "AMAZON.CancelIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.HelpIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.StopIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.NavigateHomeIntent",
                    "samples": []
                },
                {
                    "name": "ScanCubeIntent",
                    "slots": [],
                    "samples": [
                        "scan",
                        "analyze",
                        "photograph"
                    ]
                },
                {
                    "name": "CalculateSolutionIntent",
                    "slots": [],
                    "samples": [
                        "calculate",
                        "solution"
                    ]
                },
                {
                    "name": "SolveCubeIntent",
                    "slots": [],
                    "samples": [
                        "solve"
                    ]
                }
            ],
            "types": []
        }
    }
}

index.js

JavaScript
The index.js for my Alexa CraneCuber skill
/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

// This sample demonstrates sending directives to an Echo connected gadget from an Alexa skill
// using the Alexa Skills Kit SDK (v2). Please visit https://alexa.design/cookbook for additional
// examples on implementing slots, dialog management, session persistence, api calls, and more.

const Alexa = require('ask-sdk-core');
const Util = require('./util');
const Common = require('./common');

// The namespace of the custom directive to be sent by this skill
const NAMESPACE = 'Custom.Mindstorms.Gadget';

// The name of the custom directive to be sent this skill
const NAME_CONTROL = 'control';

const LaunchRequestHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'LaunchRequest';
    },
    handle: async function(handlerInput) {

        let request = handlerInput.requestEnvelope;
        let { apiEndpoint, apiAccessToken } = request.context.System;
        let apiResponse = await Util.getConnectedEndpoints(apiEndpoint, apiAccessToken);
        if ((apiResponse.endpoints || []).length === 0) {
            return handlerInput.responseBuilder
            .speak(`I couldn't find an EV3 Brick connected to this Echo device. Please check to make sure your EV3 Brick is connected, and try again.`)
            .getResponse();
        }

        // Store the gadget endpointId to be used in this skill session
        let endpointId = apiResponse.endpoints[0].endpointId || [];
        Util.putSessionAttribute(handlerInput, 'endpointId', endpointId);

        return handlerInput.responseBuilder
            .speak("Welcome to crane cuber, please insert a scrambled rubiks cube")
            .reprompt("Awaiting commands")
            .getResponse();
    }
};


const ScanCubeIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'ScanCubeIntent';
    },
    handle: function (handlerInput) {

        // Construct the directive with the payload containing the move parameters
        const attributesManager = handlerInput.attributesManager;
        let endpointId = attributesManager.getSessionAttributes().endpointId || [];
        const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
            {
                type: 'scan'
            });
            
        return handlerInput.responseBuilder
            .speak(`ok, first I will photograph all six sides of the cube. Then I will analyze the pictures to find the size and state of the cube`)
            .addDirective(directive)
            .reprompt("Awaiting commands")
            .getResponse();
    }
};

const CalculateSolutionIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'CalculateSolutionIntent';
    },
    handle: function (handlerInput) {

        // Construct the directive with the payload containing the move parameters
        const attributesManager = handlerInput.attributesManager;
        let endpointId = attributesManager.getSessionAttributes().endpointId || [];
        const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
            {
                type: 'calculate'
            });
            
        return handlerInput.responseBuilder
            .speak(`ok, it will take me about 30 seconds to calculate the solution for a five by five by five rubiks cube. There are 283 trevigintillion possible patterns for a five by five cube.  That is a huge number! If you had a stack of 283 trevigintillion sheets of paper it would be about 1.5 light years tall. `)
            .addDirective(directive)
            .reprompt("Awaiting commands")
            .getResponse();
    }
};

const SolveCubeIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'SolveCubeIntent';
    },
    handle: function (handlerInput) {

        // Construct the directive with the payload containing the move parameters
        const attributesManager = handlerInput.attributesManager;
        let endpointId = attributesManager.getSessionAttributes().endpointId || [];
        const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
            {
                type: 'solve'
            });
            
        return handlerInput.responseBuilder
            .speak(`will do, it will take 82 moves to solve this cube so this will take me a few minutes`)
            .addDirective(directive)
            .reprompt("Awaiting commands")
            .getResponse();
    }
};



// The SkillBuilder acts as the entry point for your skill, routing all request and response
// payloads to the handlers above. Make sure any new handlers or interceptors you've
// defined are included below. The order matters - they're processed top to bottom.
exports.handler = Alexa.SkillBuilders.custom()
    .addRequestHandlers(
        LaunchRequestHandler,
        ScanCubeIntentHandler,
        CalculateSolutionIntentHandler,
        SolveCubeIntentHandler,
        Common.HelpIntentHandler,
        Common.CancelAndStopIntentHandler,
        Common.SessionEndedRequestHandler,
        Common.IntentReflectorHandler, // make sure IntentReflectorHandler is last so it doesn't override your custom intent handlers
    )
    .addRequestInterceptors(Common.RequestInterceptor)
    .addErrorHandlers(
        Common.ErrorHandler,
    )
    .lambda();

package.json

JSON
The package.json for my Alexa CraneCuber skill
{
  "name": "agt-mindstorms",
  "version": "1.1.0",
  "description": "A sample skill demonstrating how to use AGT with Lego Mindstorms",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Amazon Alexa",
  "license": "ISC",
  "dependencies": {
    "ask-sdk-core": "^2.6.0",
    "ask-sdk-model": "^1.18.0",
    "aws-sdk": "^2.326.0",
    "request": "^2.81.0"
  }
}

util.js

JavaScript
The util.js for my Alexa CraneCuber skill
/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

'use strict';

const Https = require('https');

/**
 * Build a custom directive payload to the gadget with the specified endpointId
 * @param {string} endpointId - the gadget endpoint Id
 * @param {string} namespace - the namespace of the skill
 * @param {string} name - the name of the skill within the scope of this namespace
 * @param {object} payload - the payload data
 * @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#respond}
 */
exports.build = function (endpointId, namespace, name, payload) {
    // Construct the custom directive that needs to be sent
    // Gadget should declare the capabilities in the discovery response to
    // receive the directives under the following namespace.
    return {
        type: 'CustomInterfaceController.SendDirective',
        header: {
            name: name,
            namespace: namespace
        },
        endpoint: {
            endpointId: endpointId
        },
        payload
    };
};

/**
 * A convenience routine to add the a key-value pair to the session attribute.
 * @param handlerInput - the context from Alexa Service
 * @param key - the key to be added
 * @param value - the value be added
 */
exports.putSessionAttribute = function(handlerInput, key, value) {
    const attributesManager = handlerInput.attributesManager;
    let sessionAttributes = attributesManager.getSessionAttributes();
    sessionAttributes[key] = value;
    attributesManager.setSessionAttributes(sessionAttributes);
};

/**
 * To get a list of all the gadgets that meet these conditions,
 * Call the Endpoint Enumeration API with the apiEndpoint and apiAccessToken to
 * retrieve the list of all connected gadgets.
 *
 * @param {string} apiEndpoint - the Endpoint API url
 * @param {string} apiAccessToken  - the token from the session object in the Alexa request
 * @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#call-endpoint-enumeration-api}
 */
exports.getConnectedEndpoints = function(apiEndpoint, apiAccessToken) {

    // The preceding https:// need to be stripped off before making the call
    apiEndpoint = (apiEndpoint || '').replace('https://', '');
    return new Promise(((resolve, reject) => {

        const options = {
            host: apiEndpoint,
            path: '/v1/endpoints',
            method: 'GET',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': 'Bearer ' + apiAccessToken
            }
        };

        const request = Https.request(options, (response) => {
            response.setEncoding('utf8');
            let returnData = '';
            response.on('data', (chunk) => {
                returnData += chunk;
            });

            response.on('end', () => {
                resolve(JSON.parse(returnData));
            });

            response.on('error', (error) => {
                reject(error);
            });
        });
        request.end();
    }));
};

common.js

JavaScript
The common.js for my Alexa CraneCuber skill
/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

'use strict'

const Alexa = require('ask-sdk-core');

const HelpIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.HelpIntent';
    },
    handle(handlerInput) {
        const speakOutput = 'You can say hello to me! How can I help?';

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt(speakOutput)
            .getResponse();
    }
};
const CancelAndStopIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && (Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.CancelIntent'
                || Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.StopIntent');
    },
    handle(handlerInput) {
        const speakOutput = 'Goodbye!';
        return handlerInput.responseBuilder
            .speak(speakOutput)
            .getResponse();
    }
};
const SessionEndedRequestHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'SessionEndedRequest';
    },
    handle(handlerInput) {
        // Any cleanup logic goes here.
        return handlerInput.responseBuilder.getResponse();
    }
};

// The intent reflector is used for interaction model testing and debugging.
// It will simply repeat the intent the user said. You can create custom handlers
// for your intents by defining them above, then also adding them to the request
// handler chain below.
const IntentReflectorHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest';
    },
    handle(handlerInput) {
        const intentName = Alexa.getIntentName(handlerInput.requestEnvelope);
        const speakOutput = `You just triggered ${intentName}`;

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt("I don't understand this command, try again")
            .getResponse();
    }
};

// Generic error handling to capture any syntax or routing errors. If you receive an error
// stating the request handler chain is not found, you have not implemented a handler for
// the intent being invoked or included it in the skill builder below.
const ErrorHandler = {
    canHandle() {
        return true;
    },
    handle(handlerInput, error) {
        console.log(`~~~~ Error handled: ${error.stack}`);
        const speakOutput = `Sorry, I had trouble doing what you asked. Please try again.`;

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt(speakOutput)
            .getResponse();
    }
};

// The request interceptor is used for request handling testing and debugging.
// It will simply log the request in raw json format before any processing is performed.
const RequestInterceptor = {
    process(handlerInput) {
        let { attributesManager, requestEnvelope } = handlerInput;
        let sessionAttributes = attributesManager.getSessionAttributes();

        // Log the request for debug purposes.
        console.log(`=====Request==${JSON.stringify(requestEnvelope)}`);
        console.log(`=========SessionAttributes==${JSON.stringify(sessionAttributes, null, 2)}`);
    }
};

module.exports = {
    HelpIntentHandler,
    CancelAndStopIntentHandler,
    SessionEndedRequestHandler,
    IntentReflectorHandler,
    ErrorHandler,
    RequestInterceptor
    };

rubiks-cube-tracker

This is software that I wrote, it is open source and is available on github. rubiks-cube-tracker analyzes a photograph or video feed of a rubiks cube and locates the cube in the image. It then locates all of the "squares" of the cube in the image and extracts the RGB (red, green, blue) values for each square. Here is a youtube video of this software in action: https://www.youtube.com/watch?v=3tWnl9rLnfE And a blog article I wrote about how it works: http://programmablebrick.blogspot.com/2017/02/rubiks-cube-tracker-using-opencv.html

rubiks-color-resolver

This is software that I wrote, it is open source and is available on github. rubiks-color-resolver accepts a list of the RGB (red, green, blue) values for all squares on the cube and determines the exact color (white, yellow, red, orange, blue or green) of each square. This determines the exact state of the cube which will be needed by the solver to compute a solution. I wrote a blog article about how it works, just scroll down to the "Color Extraction" section http://programmablebrick.blogspot.com/2017/02/rubiks-cube-tracker-using-opencv.html

rubiks-cube-NxNxN-solver

This is software that I wrote, it is open source and is available on github. To my knowledge rubiks-cube-NxNxN-solver is the only open source NxNxN solver in the world. It is capable of solving any size rubiks cube, I have tested up to a 17x17x17!! It was about 8 months of work to progress from not knowing how to solve a rubiks cube at all to being able to solve NxNxN.

lego-crane-cuber

This is software that I wrote, it is open source and is available on github. This is the CraneCuber program that runs on the EV3.

Credits

dwalton76

dwalton76

1 project • 5 followers

Comments