PoE Automatic Potion Drink!! If Hit point is half…


Warning

This posting is for educational purposes only. Never abuse, it gets misused.

Hello this is lifeonroom 

I added an upgrade to my last post and this time I used OpenCV to create a Python code that automatically takes potions when your health or mana is half-baked! Note that the feature only works on the main monitor. If you’re using dual monitors, you won’t be able to work on sub monitors!! 

First, let’s learn about the files that were added or changed before entering.

  • drinkportion.py – Changed to press a specific key as a result of the change and image processing. 
  • config.cfg – Added options for changes, video processing, and the keyboard value to press at critical point Trigger. 
  • cvFunc.py – It is about additional and video processing.

Now let’s find out which libraries you need to install!

1. Library

pip install opencv-python
pip install pillow
pip install numpy
pip install mss
pip install pyautogui==0.9.38

 

Here’s a certain version of pyautogui! For the latest version of pyautogui, the installation in Hangul windows is error…. So you need to install the version mentioned above to make it work normally.

2. config.cfg

[portion_drink_delay]
delay = 0.1
delay_triggered = 1
[portion_key]
k_1 = 2,3,4,5
# m_right = 5
m_middle = 4
s_hp = 1
s_mp = 5
[screen]
bgr_lower = {'sheild': (89, 132, 72), 'mp': (109, 47, 13), 'hp': (33, 94, 30)}
bgr_upper = {'sheild': (169, 199, 137), 'mp': (158, 71, 32), 'hp': (47, 147, 33)}
hpmp_region = (1818, 655, 201, 38)
#hpmp_region = (911, 319, 101, 18) #full hd
#hpmp_region = (906, 314, 107, 20) # full hd whole screen mode
hpmp_trigger = 0.5

 

Let’s take a look at the config.cfg file giving the program options 

The screen section, the delay_triggered option in the part_drink_delay section, and the portion_key section have been added to the s_hp and s_mp options. The meaning of each option is the same as:

  • bgr_lower, bgr_upper: BGR minimum and maximum value of sheild, mp, hp bar. 
  • hpmp_region: Where sheild, mp, and hp bars exist. 
  • hpmp_trigger: mp, hp, it’s a shame to take a potion when it’s dropped. In this case, if you have 50% left, you will take the potion. 
  • delay_triggered: mp, how much Delay should be placed and potion when the hp level is under hpmp_trigger. In this case, it is 1 second. 
  • s_hp, s_mp: Hp, mp is about which key to press when going down under hpmp_trigger. In this case, the hp is 1 key and mp is 2. 

Note that the value of bgr_lower, bgr_uppwer works well with the above values that I’ve tested with 3 monitors. So i think you can use the above value without getting it aside!

2. Code 

In the last post, a file called cvFunc.py was added. The file detects the character’s blood and manatong and tells you when to take the potion! In addition, we’ve implemented the following features:  

  1. Get coordinate values for a specific Image area 
  2. Get the minimum and maximum channel-specific value of brg value for a particular photo 
  3. Detecting rectangles with specific BGR values in an image 
  4. Take a series of screenshots to detect blood and manatong 

 

cvFunc.py

#-*- coding: utf-8 -*-
from ast import literal_eval
import os
import time
import cv2.cv2 as cv2
import numpy as np
import pyautogui as pa
from PIL import Image
import copy
from mss import mss
class ScreenShot():
    finalRegion = 0
    def __init__(self, region=(0, 0, 0, 0)):
        if region == (0, 0, 0, 0):
            screen = np.array(pa.screenshot())
            self.img = screen[:, :, ::-1].copy()
        else:
            screen = np.array(pa.screenshot(region=region))
            self.img = screen[:, :, ::-1].copy()
        #self.overlay = self.img.copy()
        self.output = self.img.copy()
        self.drawing = False
        self.ix = 0
        self.iy = 0
        sz = pa.size()
        coord = (0, 0, sz[0], sz[1])
        self.scrCenter = pa.center(coord)
        self.mouseCoord = pa.position()
    def drawBlueRect(self, event, x, y, flags, param):
        #global ix, iy, drawing, mode, output
        if event == cv2.EVENT_LBUTTONDOWN:  # 마우스를 누른 상태
            self.drawing = True
            self.ix, self.iy = x, y
        elif event == cv2.EVENT_MOUSEMOVE:  # 마우스 이동
            if self.drawing == True:  # 마우스를 누른 상태 일경우
                self.output = self.img.copy()
                cv2.rectangle(self.output, (self.ix, self.iy), (x, y), (255, 0, 0), 2)
                pass
        elif event == cv2.EVENT_LBUTTONUP:
            self.drawing = False;  # 마우스를 때면 상태 변경
            cv2.rectangle(self.output, (self.ix, self.iy), (x, y), (255, 0, 0), 2)
            startX, endX = self.ix, x
            startY, endY = self.iy, y
            if x > self.ix :
                startX, endX = self.ix, x
            else:
                startX, endX = x, self.ix
            if y > self.iy:
                startY, endY = self.iy, y
            else:
                startY, endY = y, self.iy
            self.finalRegion = (startX, startY, endX - startX, endY - startY)
    def partScreenShot(self):
        cv2.namedWindow('image', cv2.WINDOW_NORMAL)
        cv2.setWindowProperty("image", cv2.WND_PROP_FULLSCREEN, cv2.WINDOW_FULLSCREEN);
        cv2.setMouseCallback('image', self.drawBlueRect)
        while True:
            cv2.imshow('image', self.output)
            k = cv2.waitKey(1) & 0xFF
            if k == 27:  # esc를 누르면 종료
                break
        cv2.destroyAllWindows()
        print("Screen Shot Region(x, y, w, h): ", self.finalRegion)
        return self.finalRegion
    def saveImage(self, fileName, region=()):
        if len(region) == 0:
            x, y, w, h = self.finalRegion
        else:
            x, y, w, h = region
        croped_img = self.img[y:y+h, x:x+w]
        path = os.path.dirname(os.path.realpath(__file__)) + '\\' + str(fileName)
        cv2.imwrite(path, croped_img)
class ScreenProcess():
    drinkTime = {}
    HPMPText = 'Text'
    recogCnt = 0
    def __init__(self, dict_bgr_lower, dict_bgr_upper, trigger=0.5, delayTriggered = 3, region = (0, 0, 0, 0)):
        self.lower = dict_bgr_lower
        self.upper = dict_bgr_upper
        for key, _ in self.upper.items():
            self.drinkTime[key] = time.time()
        self.x, self.y, self.w, self.h = region
        self.mon = {'top': self.y, 'left': self.x, 'width': self.w, 'height': self.h}
        self.sct = mss()
        self.delayTriggered = delayTriggered
        self.trigger = trigger
    def checkHPMP(self, frame, show=0, videoShow = 0):
        rects = []
        triggered = []
        for key, value in self.upper.items():
            img = copy.copy(frame)
            height, width = img.shape[:2]
            #print(height, width)
            img = cv2.GaussianBlur(img, (11, 11), 0)
            mask = cv2.inRange(img, self.lower[key], self.upper[key])
            res = cv2.bitwise_and(img, img, mask=mask)
            gray = cv2.cvtColor(res, cv2.COLOR_BGR2GRAY)
            #ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY)
            #kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 5))
            #erosion = cv2.erode(thresh, kernel, iterations=1)
            cnts = cv2.findContours(gray.copy(), cv2.RETR_EXTERNAL,
                                    cv2.CHAIN_APPROX_SIMPLE)[-2]
            for cnt in cnts:
                approx = cv2.approxPolyDP(cnt, 0.1 * cv2.arcLength(cnt, True), True)
                if (len(approx) >= 2):
                    # print("approx:", len(approx), "CountourArea: ", cv2.contourArea(cnt))
                    (x, y, w, h) = cv2.boundingRect(approx)
                    rects.append((x, y, w, h))
                    currentHPMP = w/(width * 1.0)
                    if currentHPMP < self.trigger and x < 10:
                        if (time.time() - self.drinkTime[key] > self.delayTriggered and key != 'sheild'):
                            self.drinkTime[key] = time.time()
                            showText = key + ' need portion!' + str(self.recogCnt)
                            print(showText)
                            self.recogCnt += 1
                            self.HPMPText = showText
                            triggered.append(key)
                    break
        if show == 1:
            showImg = copy.copy(frame)
            textImg = np.zeros((height * 2, width*2, 3), np.uint8)
            cv2.putText(textImg, self.HPMPText, (5, height+ 5), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), lineType=cv2.LINE_AA)
            for detectRect in rects:
                (x, y, w, h) = detectRect
                cv2.rectangle(showImg, (x, y), (x + w, y + h), (255, 0, 255), 2)
            viewImage = np.hstack([frame, showImg])
            finalView = np.vstack([viewImage, textImg])
            cv2.imshow("Frame Result", finalView)
            cv2.resizeWindow('Frame Result', 300, 300)
            #cv2.imshow("OneFrame", np.hstack([thresh, gray, erosion]))
            if videoShow == 0:
                cv2.waitKey(0)
        if show == 1 and videoShow == 0:
            cv2.destroyAllWindows()
        return triggered
    def screenProcess(self, showVideo=0):
        self.sct.grab(self.mon)
        img = Image.frombytes('RGB', (self.w, self.h), self.sct.grab(self.mon).rgb)
        frame = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
        return self.checkHPMP(frame, show=showVideo, videoShow=showVideo)
        # if cv2.waitKey(25) & 0xFF == ord('q'):
        #     cv2.destroyAllWindows()
        #     break
def checkColorMinMax(listImageName):
    cmin = {}
    cmax = {}
    for imageName in listImageName:
        frame = cv2.imread(imageName)
        frame = cv2.GaussianBlur(frame, (11, 11), 0)
        max_ch = (np.amax(frame[:, :, 0]), np.amax(frame[:, :, 1]), np.amax(frame[:, :, 2]))
        min_ch = (np.amin(frame[:, :, 0]), np.amin(frame[:, :, 1]), np.amin(frame[:, :, 2]))
        specName = imageName.split('.')[0]
        cmin[specName] = min_ch
        cmax[specName] = max_ch
    print("lower = ", cmin)
    print("upper = ", cmax)
if __name__=="__main__":
    import configparser
    configFile = os.path.dirname(os.path.realpath(__file__)) + '\\' + 'config.cfg'
    config = configparser.ConfigParser()
    config.read(configFile)
    delay_triggered = int(config['portion_drink_delay']['delay_triggered'])
    brg_lower = literal_eval(config['screen']['bgr_lower'])
    bgr_upper = literal_eval(config['screen']['bgr_upper'])
    hpmp_region = literal_eval(config['screen']['hpmp_region'])
    hpmp_trigger = float(config['screen']['hpmp_trigger'])
    test = 1
    # -------- ScreenSize ------------
    if test == 1:
        ip = ScreenShot()
        ip.partScreenShot()
    # -------- Check Color BGR min max ------------
    elif test == 2:
        checkColorMinMax(["sheild.png", "mana.png", "health.png"])
    # -------- detect HP/MP on image ------------
    elif test == 3:
        frame = cv2.imread("testpoe.png")
        sp = ScreenProcess(brg_lower, bgr_upper, hpmp_trigger, delay_triggered, hpmp_region)
        sp.checkHPMP(frame, show=1)
    # -------- detect HP/MP on continuous Screen ------------
    elif test == 4:
        sp = ScreenProcess(brg_lower, bgr_upper, hpmp_trigger, delay_triggered, hpmp_region)
        while True:
            sp.screenProcess(showVideo=1)
            if cv2.waitKey(25) & 0xFF == ord('q'):
                cv2.destroyAllWindows()
                break

 

 

If you look at the Main below, you can change the test variable to test the features mentioned above 1-4 times. !! Please refer to the video of the behavior of the code. 

drinkPortion.py

import mouse as mo
import keyboard as keys
import time
import configparser
import os
from ast import literal_eval
import cv2.cv2 as cv2
#try:
import keyEvent as ke
# import screen process module 
import cvFunc as cvf
# except:
#     import POE.keyEvent as ke
#     import POE.cvFunc as cvf
try:
    import serialFunc as sf
    ser = sf.ExternalHID('COM16')
except Exception as e:
    print(e)
    print("Hardware Macro Disabled")
def generateKeyEvent(val, key_s, delay):
    if val == True:
        for outVal in key_s:
            # hardware macro
            #ser.keyboardInput(outVal)
            # software macro
            ke.press(outVal)
            time.sleep(delay)
def drinkPortionWithInput(listDevKeyOutVal, delay=0.001, sProcess=None):
    listKeyState = [0] * len(listDevKeyOutVal)
    triggereds = []
    while True:
        if sProcess != None:
      # screenProcess if triggered return data
            triggereds = sProcess.screenProcess(showVideo=1)
            if cv2.waitKeyEx(25) == ord('`'):
                cv2.destroyAllWindows()
                break
        for idx, dictDevVal in enumerate(listDevKeyOutVal):
            keyormo = list(dictDevVal.keys())[0]
            generateKeys = dictDevVal[keyormo].split(',')
            # [0] = k is keyboard [1] = pushed key
            if keyormo.split('_')[0].strip() == 'k':
                value = keys.is_pressed(keyormo.split('_')[1].strip())
                if listKeyState[idx] != value:
                    generateKeyEvent(value, generateKeys, delay)
                    listKeyState[idx] = value
            # [0] = m is mouse [1] = pushed mouse btn
            elif keyormo.split('_')[0].strip() == 'm':
                possList = [mo.LEFT, mo.RIGHT, mo.MIDDLE]
                try:
                    possList.index(keyormo.split('_')[1].strip())
                except Exception as e:
                    continue
                value = mo.is_pressed(keyormo.split('_')[1].strip())
                if listKeyState[idx] != value:
                    generateKeyEvent(value, generateKeys, delay)
                    listKeyState[idx] = value
            elif keyormo.split('_')[0].strip() == 's':
                stat = keyormo.split('_')[1].strip()
        # drink potion if HP, MP below specific percentage
                for triggered in triggereds:
                    if triggered == stat:
                        generateKeyEvent(True, generateKeys, delay)
if __name__=="__main__":
    configFile = os.path.dirname(os.path.realpath(__file__)) + '\\' + 'config.cfg'
    config = configparser.ConfigParser()
    config.read(configFile)
    itemList = []
    for option in config.options('portion_key'):
        itemList.append({option:config['portion_key'][option]})
    drinkDelay = float(config['portion_drink_delay']['delay'])
    delay_triggered = int(config['portion_drink_delay']['delay_triggered'])
    brg_lower = literal_eval(config['screen']['bgr_lower'])
    bgr_upper = literal_eval(config['screen']['bgr_upper'])
    hpmp_region = literal_eval(config['screen']['hpmp_region'])
    hpmp_trigger = float(config['screen']['hpmp_trigger'])
    print("macro start drink delay  %s " % str(drinkDelay))
  # make screenProcess instance
    sp = cvf.ScreenProcess(brg_lower, bgr_upper, hpmp_trigger, delay_triggered, hpmp_region)
    drinkPortionWithInput(itemList, drinkDelay, sp)

 

 

The new drinkPortion.py. I’ve commented on the main part. 

3. Run 

After coding, enable the virtual environment in the CMD environment and then python drinkPortion the code will run. And if you want to keep the video processing window open, you can change it to showVideo = 0. 

Yep today’s post is here. I’m going to act 4 mill now, but i’ve done my way to build it, and i’ve almost folded the game. 

 

You may also like...

Leave a Reply

Your email address will not be published.