我是新到吉姆的,我试图做一个简单的学习方案,但就一些(weird)而言,它赢得的只是让我摆脱了那部分内容(它正在永远进行)。

我的日程安排如下:

import gymnasium as gym
import numpy as np

env = gym.make("MountainCar-v0", render_mode="human")

LEARNING_RATE = 0.1
DISCOUNT = 0.95
EPISODES = 25000

SHOW_EVERY = 500

DISCRETE_OS_SIZE = [20] * len(env.observation_space.low)
discrete_os_win_size = (env.observation_space.high - env.observation_space.low) / DISCRETE_OS_SIZE

q_table = np.random.uniform(low=-2, high=0, size=(DISCRETE_OS_SIZE + [env.action_space.n]))


def get_discrete_state(state):
    discrete_state = (state - env.observation_space.low) / discrete_os_win_size
    return tuple(discrete_state.astype(int))


for episode in range(EPISODES):
    if episode % SHOW_EVERY == 0:
        render = True
    else:
        render = False
    
    print("Episode:", episode)
    
    discrete_state = get_discrete_state(tuple(env.reset()[0].astype(int)))
    
    done = False
    while not done:
        action = np.argmax(q_table[discrete_state])
        new_state, reward, terminated, truncated, _ = env.step(action)
        done = truncated or terminated
        
        new_discrete_state = get_discrete_state(new_state)
        
        # Rendering the episode
        # (Even removing this part does not help)
        if render:
            env.render()

        if not done:
            # Updating the Q-table
            max_future_q = np.max(q_table[new_discrete_state])
            current_q = q_table[discrete_state + (action, )]
            new_q = (1-LEARNING_RATE)* current_q + LEARNING_RATE * (reward + DISCOUNT * max_future_q)
            
            q_table[discrete_state + (action, )] = new_q
       
        # If the car made it to the goal
        elif new_state[0] >= env.unwrapped.goal_position:
            q_table[discrete_state + (action, )] = 0
            print("MADE IT ON EPISODE:", episode)
        discrete_state = new_discrete_state
    
env.close()

我尝试:

• removing the env.render() part: did not work
• removing the discrete_state at the start and replace it by the default value (13, 10) by hand: kinda worked (episodes not rendering but also not the ones when render is True)