未能将 Cookie 从 Selenium webdriver 传递到 clojure 中的 httpclient cookie store

cookies selenium java clojure

2022-09-04 23:15:09

我想使用Selenium登录到一个网站，然后将cookie导出到httpclient。

(defn convert-selemium-cookie-to-httpclient-cookie [x]
  (let [sf (SimpleDateFormat. "yyyy-MM-dd")
        fake-date (.parse sf "2018-08-06")]
    (doto
        (BasicClientCookie. (:name x) (:value x))
        (.setDomain (:domain x))
        (.setPath (:path x))
        (.setExpiryDate (if (:expiry x) (:expiry x) fake-date))
        (.setSecure (:secure? x)))))

(defn add-selenium-cookies-to-httpclient-cookie-store [x]
  (let [cs (clj-http.cookies/cookie-store)
        http-cookies (map convert-selemium-cookie-to-httpclient-cookie x)]
    (mapv (fn[x] (.addCookie cs x)) http-cookies)
    cs))

(def driver (new-driver {:browser :firefox}))
(def a (login driver ...)) ;; login
(def c (cookies driver)) ;; get the cookies
(def cs (add-selenium-cookies-to-httpclient-cookie-store c))
(println (client/get "some web site"
                     {:cookie-store cs
                      :client-params {"http.useragent"
                                      "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.20 (KHTML, like Gecko) Chrome/11.0.672.2 Safari/534.20"}}))

虽然我在网络驱动程序中成功登录，并且该网站在会话之间保留cookie，但在将cookie复制到cookie存储后，仍然返回未登录的页面。我在python中尝试过类似的东西，它可以成功地将cookie从webdriver传输到请求（python的http客户端）。不确定java/clojure的问题在哪里。client/get

答案 1

尚未指定依赖项的版本。我用以下内容测试了您的代码：

[org.clojure/clojure "1.8.0"]
[clj-webdriver "0.7.2"]
[org.seleniumhq.selenium/selenium-server "3.0.0-beta4"]

Firefox 48.0.2
Mac OS Sierra 10.12

我的答案末尾的REPL会话表明它有效（我曾经测试过它）。https://httpbin.org/

要进行故障排除，我会检查以下内容：

打印从驱动程序返回的 cookie 或检查浏览器控制台，以查看登录页面的响应是否包含设置正确 Cookie 的标题
打印您的 http 客户端 Cookie 存储包含的内容
为 http-client 启用调试日志记录（或在服务器中启用调试日志记录），以查看发送到受登录保护的页面的实际请求，以及请求是否包含登录页面设置的 Cookie。

在REPL会话中，您可以看到从硒返回的cookie包含第一个请求设置的cookie。它们也存在于 http-client cookie 存储区中，并且它们由 URL 正确返回，指示它们已通过 http-client 与请求一起发送。https://httpbin.org/cookies

(require '[clj-webdriver.taxi :as taxi])
(require '[clj-http.client :as client])
(require '[clj-http.cookies :as cookies])
(require '[clojure.pprint :refer [pprint]])
(import java.text.SimpleDateFormat)
(import org.apache.http.impl.cookie.BasicClientCookie)

(defn convert-selemium-cookie-to-httpclient-cookie [x]
  (let [sf (SimpleDateFormat. "yyyy-MM-dd")
        fake-date (.parse sf "2018-08-06")]
    (doto
      (BasicClientCookie. (:name x) (:value x))
      (.setDomain (:domain x))
      (.setPath (:path x))
      (.setExpiryDate (if (:expiry x) (:expiry x) fake-date))
      (.setSecure (:secure? x)))))

(defn add-selenium-cookies-to-httpclient-cookie-store [x]
  (let [cs (cookies/cookie-store)
        http-cookies (map convert-selemium-cookie-to-httpclient-cookie x)]
    (mapv (fn [x] (.addCookie cs x)) http-cookies) cs))

(def cookie-name (str "c1" (System/currentTimeMillis)))
(def cookie-value (str "v1" (System/currentTimeMillis)))

(pprint cookie-name)
;; "c11475935066134"

(pprint cookie-value)
;; "v11475935066814"

(def driver (taxi/new-driver {:browser :firefox}))

(taxi/to driver (format "https://httpbin.org/cookies/set?%s=%s" cookie-name cookie-value))

(def selenium-cookies (taxi/cookies driver))

(pprint selenium-cookies)
;; #{{:cookie
;;    #object[org.openqa.selenium.Cookie 0x4dc96ce8 "c11475935066134=v11475935066814; path=/; domain=httpbin.org"],
;;    :name "c11475935066134",
;;    :value "v11475935066814",
;;    :path "/",
;;    :expiry nil,
;;    :domain "httpbin.org",
;;    :secure? false}}

(def http-client-cookie-store (add-selenium-cookies-to-httpclient-cookie-store selenium-cookies))

(pprint http-client-cookie-store)
;; #object[org.apache.http.impl.client.BasicCookieStore 0x6dfa86f5 "[[version: 0][name: c11475935066134][value: v11475935066814][domain: httpbin.org][path: /][expiry: Mon Aug 06 00:00:00 CEST 2018]]"]

(def http-client-response
  (client/get
    "https://httpbin.org/cookies"
    {:cookie-store http-client-cookie-store}))

(pprint http-client-response)
;; {:status 200,
;;  :headers
;;  {"Server" "nginx",
;;   "Date" "Sat, 08 Oct 2016 13:58:01 GMT",
;;   "Content-Type" "application/json",
;;   "Content-Length" "64",
;;   "Connection" "close",
;;   "Access-Control-Allow-Origin" "*",
;;   "Access-Control-Allow-Credentials" "true"},
;;  :body
;;  "{\n  \"cookies\": {\n    \"c11475935066134\": \"v11475935066814\"\n  }\n}\n",
;;  :request-time 1001,
;;  :trace-redirects ["https://httpbin.org/cookies"],
;;  :orig-content-encoding nil}

答案 2

你试过硒请求吗？https://pypi.python.org/pypi/selenium-requests/