miércoles, 13 de marzo de 2019

Sikulix(1): Installation for Java

0. Introduction

Sikuli seems to be an option for using certificate authentication in browsers due to the fact that the certificate selection is an OS task (see Stackoverflow) and not a browser specific action. So Selenium can hardly be useful alone for doing all tasks automatically.

Perhaps, a better solution can be select "by hand" the required certificate and tell selenium to wait until the desired page is loaded (the subsequent page after the certificate selection). See StackOverflow.



1. Installation

Reading Sikuli's docs, related to Ubuntu, it is advisable to use Ubuntu 18.04 (last releases) for avoiding problems with OpenCV installation.

I have not been able to install OpenCV version 3.2 on Ubuntu 16.04 so I have upgraded to 18.04.

These are the main prerequisites.


1
2
3
4
5
6
7
8
#1. Install libcanberra
sudo apt install libcanberra-gtk0 libcanberra-gtk-module

#2Install libopencv3.2-java
sudo apt install libopencv3.2-java

#3 Make a link with a known name for Sikuli
sudo ln -s /usr/lib/jni/libopencv_java320.so /usr/lib/libopencv_java.so

It is important to install Tesseract (version 4.1.1). Bharath Sivakumar explains how to do this task.  He also explains how to use OCR programmatically!. It is very good!

Basically (Please follow the last link!):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#1. Check Tesseract version
tesseract --version

#2. Install prerequisite "leptonica2
sudo apt-get install -y libleptonica-dev

#3.Install dependencies
sudo apt-get update -y
sudo apt-get install automake
sudo apt-get install -y pkg-config
sudo apt-get install -y libsdl-pango-dev
sudo apt-get install -y libicu-dev
sudo apt-get install -y libcairo2-dev
sudo apt-get install bc

#get the source code and unzip it
wget https://github.com/tesseract-ocr/tesseract/archive/4.1.1.zip
unzip 4.1.1.zip

#go to tesseract-4.1.1 directory and execute
./autogen.sh
./configure
make
sudo make install
sudo ldconfig
make training
sudo make training-install

#test tesseract version
tesseract --version



2. Updating the pom.xml from the last project



  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>ximodante</groupId>
  <artifactId>HTMLFillForms</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>HTMLFillForms</name>
  <description>HTMLFillForms</description>
  
  <repositories>
    <repository>
      <!-- See https://github.com/RaiMan/SikuliX1 -->
      <id>repo-sikuli</id>
      <name>sonatype nightky bluids for sikuli</name>
      <url>http://oss.sonatype.org/content/groups/public</url>
    </repository>
  </repositories>
  
  <properties>
    <cxf.version>3.2.6</cxf.version>
    <jax.version>2.3.0</jax.version>
    <lombok.version>1.18.4</lombok.version>
    <jackson.version>2.9.6</jackson.version>
    <h2.version>1.4.196</h2.version>
    <hibernate.version>5.2.16.Final</hibernate.version>
    <selenium.version>3.141.59</selenium.version>
    <poi.version>4.0.1</poi.version>
    <sikulix.version>2.0.4</sikuli.version>
    
        
    <failOnMissingWebXml>false</failOnMissingWebXml>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <!-- Java version-->
    <java.version>13</java.version>
  </properties>  
  
  <dependencies>
  
    <!-- https://mvnrepository.com/artifact/org.projectlombok/lombok -->
    <dependency>
      <groupId>org.projectlombok</groupId>
      <artifactId>lombok</artifactId>
      <version>${lombok.version}</version>
      <scope>provided</scope>
    </dependency>
    
    <!-- https://mvnrepository.com/artifact/org.apache.commons/commons-lang3 -->
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>commons-lang3</artifactId>
      <version>3.8.1</version>
    </dependency>
    

    <!--  HTML Parser  -->
    <!--  See https://www.mkyong.com/java/how-to-automate-login-a-website-java-example/ -->
    <!-- https://mvnrepository.com/artifact/org.jsoup/jsoup -->
    <!-- 
    <dependency>
      <groupId>org.jsoup</groupId>
      <artifactId>jsoup</artifactId>
      <version>1.11.3</version>
    </dependency>
     -->

    <!-- Simulate a browser to fill forms !!! -->
    <!--  seehttps://stackoverflow.com/a/1383097/7704658   -->
    <!-- https://mvnrepository.com/artifact/net.sourceforge.htmlunit/htmlunit -->
    <!-- 
    <dependency>
      <groupId>net.sourceforge.htmlunit</groupId>
      <artifactId>htmlunit</artifactId>
      <version>2.33</version>
    </dependency>
     -->
     
    <!-- Seleniun -->
    <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    
    <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-server -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-server</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    
    <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-firefox-driver -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-firefox-driver</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    
    <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-chrome-driver -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-chrome-driver</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    
    
    
    <!-- Sikuli ???? -->
    <dependency>
     <groupId>com.sikulix</groupId>
     <artifactId>sikulixapi</artifactId>
     <version>${sikulix.version}</version>
    </dependency>
    
    <!--  Tess4J A Java JNA wrapper for Tesseract OCR API.  -->
    <!-- https://mvnrepository.com/artifact/net.sourceforge.tess4j/tess4j -->
    <!-- 
    <dependency>
      <groupId>net.sourceforge.tess4j</groupId>
      <artifactId>tess4j</artifactId>
      <version>4.3.1</version>
    </dependency>
    -->
    
    <!-- https://mvnrepository.com/artifact/org.bytedeco.javacpp-presets/opencv -->
    <!-- 
    <dependency>
      <groupId>org.bytedeco.javacpp-presets</groupId>
      <artifactId>opencv</artifactId>
      <version>4.0.1-1.4.4</version>
    </dependency>
    -->
  
    <!--  JPA  -->
    <!-- MS-SQL Server JDBC JTDS bo de 2013-->
    <!-- https://mvnrepository.com/artifact/net.sourceforge.jtds/jtds -->
    <dependency>
      <groupId>net.sourceforge.jtds</groupId>
      <artifactId>jtds</artifactId>
      <version>1.3.1</version>
    </dependency>
     
       
    
    <!-- JPA 2.1 Provider -->
    <!-- https://mvnrepository.com/artifact/org.hibernate/hibernate-core -->
    <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-core</artifactId>
      <version>${hibernate.version}</version>
    </dependency>
    
    <!--  END JPA -->
    
    
    <!-- For using Excel Files -->
    <!-- https://mvnrepository.com/artifact/org.apache.poi/poi -->
    <dependency>
      <groupId>org.apache.poi</groupId>
      <artifactId>poi</artifactId>
      <version>${poi.version}</version>
    </dependency>
        
    
    <!-- https://mvnrepository.com/artifact/org.apache.poi/poi-ooxml -->
    <dependency>
      <groupId>org.apache.poi</groupId>
      <artifactId>poi-ooxml</artifactId>
      <version>${poi.version}</version>
    </dependency>
    
  
    <!--BEGIN Java 9 references to JEE  not included in JDK9-->
    <!-- see http://openjdk.java.net/jeps/320-->
    
    <!-- https://mvnrepository.com/artifact/com.sun.xml.bind/jaxb-impl -->
    <dependency>
      <groupId>com.sun.xml.bind</groupId>
      <artifactId>jaxb-impl</artifactId>
      <version>${jax.version}</version>
    </dependency>
        
    <!-- https://mvnrepository.com/artifact/com.sun.xml.bind/jaxb-core -->
    <dependency>
      <groupId>com.sun.xml.bind</groupId>
      <artifactId>jaxb-core</artifactId>
      <version>${jax.version}</version>
    </dependency>
    
    
    <dependency>
      <groupId>com.sun.activation</groupId>
      <artifactId>javax.activation</artifactId>
      <version>1.2.0</version>
    </dependency>
    
    <dependency>
      <groupId>com.sun.xml.ws</groupId>
      <artifactId>jaxws-ri</artifactId>
      <version>${jax.version}</version>
      <type>pom</type>
    </dependency>
        
    <dependency>
      <groupId>com.sun.xml.bind</groupId>
      <artifactId>jaxb-ri</artifactId>
      <version>${jax.version}</version>
      <type>pom</type>
    </dependency>
    
    <!-- End of Dependencies required for JAVA 9!!!! -->
    
  </dependencies>
  
  <!-- COMPILER PLUGIN -->
  <!-- This is OK  http://crunchify.com/how-to-create-build-java-project-including-all-dependencies-using-maven-maven-resources-maven-dependency-maven-jar-plugin-tutorial/
  -->
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.8.0</version>
          <configuration>
            <!-- release>10</release -->
            <source>10</source>
            <target>10</target>
            <showWarnings>true</showWarnings>
            <showDeprecation>true</showDeprecation>
          </configuration>
          
        </plugin>
      </plugins>
    </pluginManagement>
 
    <plugins>
      <plugin>
        <artifactId>maven-resources-plugin</artifactId>
        <version>3.1.0</version>
        <executions>
          <execution>
            <id>copy-resources</id>
            <phase>validate</phase>
            <goals>
              <goal>copy-resources</goal>
            </goals>
            <configuration>
              <outputDirectory>${basedir}/target/MyFolder</outputDirectory>
              <resources>
                <resource>
                  <directory>resources</directory>
                  <filtering>true</filtering>
                </resource>
              </resources>
            </configuration>
          </execution>
        </executions>
      </plugin>
 
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>3.1.1</version>
        <executions>
          <execution>
            <id>copy-dependencies</id>
            <phase>prepare-package</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>${project.build.directory}/MyFolder/lib</outputDirectory>
              <overWriteReleases>false</overWriteReleases>
              <overWriteSnapshots>false</overWriteSnapshots>
              <overWriteIfNewer>true</overWriteIfNewer>
            </configuration>
          </execution>
        </executions>
      </plugin>
   
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.1.0</version>
        <configuration>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <classpathPrefix>lib/</classpathPrefix>
              <mainClass>u.request.MyExecutableClass</mainClass>
            </manifest>
            <manifestEntries>
              <Class-Path>.</Class-Path>
            </manifestEntries>
          </archive>
 
          <finalName>MyFolder/MyJars</finalName>
        </configuration>
      </plugin>
    </plugins>
  
  </build>
  
</project>


The new changes are displayed on a yellow background:

Lines 9-16 indicates the new maven repository to find the latest Sikuli version as states in the "README.md" file in the  Sikuli github.

Line 27 indicates the Sikulix version

Lines 107-112 indicates the dependency to add to the project.


3. The easy example

This is a similar example to all tutorials!!!!
  1. Right-click on the desktop window and create a new folder and name it "TestFolder".
  2. Use Shutter screenshot for selecting the image of the folder. As this one
  3. Name this image as "TestFolderImage.png" 
  4. Put it in a folder like "/home/user/myImages"  (where "user" is your username!!!)
  5. This is the address of the patter to find in the desktop screen "/home/user/myImages/TestFolderImage.png"
  6. Let's create a Screen element and a Pattern element and let's double click on the part of the Desktop region that matches this pattern image.
Here is the code 


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
package test;

import org.sikuli.script.FindFailed;
import org.sikuli.script.Pattern;
import org.sikuli.script.Screen;

public class Sikuli02 {

  public static void main(String[] args) throws FindFailed {
    Screen s = new Screen();
    Pattern folderToFind = new Pattern("/home/ximo/myImages/TestFolderImage.png");
    // Finds the image on the desktop and doubleclick on it
    s.doubleClick(folderToFind);  
  }
}





Publicado por en

No hay comentarios:

Publicar un comentario

Suscribirse a: Enviar comentarios (Atom)