06 Comparison with SHAP

We will compare Shapley value computation between shapley-numba and SHAP.

Remember to pip install shapley-numba[shap].

import numpy as np
import shap

from shapley_numba.examples import GloveGame
from shapley_numba.shapley import shapley

We will define our game.

Recall that in shapley_numba, number of players is often an external parameter that needs to be supplied to computations.

The following defines a glove market game of 12 players with 3 left gloves sellers (and 9 right gloves sellers).

num_players = 12
game = GloveGame(3)

explainer = shap.explainers.Exact(
    lambda x: [game.value(c) for c in x], np.zeros((1, num_players), dtype=np.int32)
)

explainer(np.ones((1, num_players), dtype=np.int32))

.values =
array([[0.85909091, 0.85909091, 0.85909091, 0.0469697 , 0.0469697 ,
        0.0469697 , 0.0469697 , 0.0469697 , 0.0469697 , 0.0469697 ,
        0.0469697 , 0.0469697 ]])

.base_values =
array([0.])

.data =
array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]], dtype=int32)

shapley(game, num_players)

array([0.85909091, 0.85909091, 0.85909091, 0.0469697 , 0.0469697 ,
       0.0469697 , 0.0469697 , 0.0469697 , 0.0469697 , 0.0469697 ,
       0.0469697 , 0.0469697 ])

Performance comparison

Let’s increase the number of players to compare the performance of two packages.

high_number_of_players = 20

explainer = shap.explainers.Exact(
    lambda x: [game.value(c) for c in x],
    np.zeros((1, high_number_of_players), dtype=np.int32),
)

%%timeit
explainer(np.ones((1, high_number_of_players), dtype=np.int32), max_evals=None)

2.16 s ± 30.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%%timeit
shapley(game, high_number_of_players)

67.6 ms ± 729 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)

While this is faster, there is a slight of hand - the adapter of our game for SHAP is a pure python loop. Let’s try to write it in numba.

import numba


def shap_game_adapter(game):
    jitted_game = game.jitted_game

    @numba.njit
    def _value_of_coalitions(jg, coalitions):
        ln = len(coalitions)
        result = np.zeros(ln)
        for i, c in enumerate(coalitions):
            result[i] = jg.value(c)
        return result

    def value_of_coalitions(coalitions):
        return _value_of_coalitions(jitted_game, coalitions)

    return value_of_coalitions

shap_glove = shap_game_adapter(game)
explainer = shap.explainers.Exact(
    shap_glove,
    np.zeros(
        (
            1,
            high_number_of_players,
        ),
        dtype=np.int32,
    ),
)

%%timeit
explainer(np.ones((1, high_number_of_players), dtype=np.int32), max_evals=None)

916 ms ± 8.56 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)