nodejs简易代理服务器

Posted on 2021-05-10 11:14:39
Comments: 0
Author: 可乐小可爱メ
1. 背景 

请求三方服务,因为服务器在海外, 网络延迟高, 实际部署上线后, 各种接口全都超时,严重影响实用效率.


2. 技术设计

  通过加一层 proxy server 做转发

        user --> client. --> proxy server --> business server .    

3. 代码实现

    3.1 代码结构

    

    3.2 

        proxy.js

const express = require("express");
const { createProxyMiddleware } = require("http-proxy-middleware");
const target = require("./.TARGET");

const app = express();
app.get("/", (req, res) => {
res.send("express 20010 home");
});

app.use(
"/api",
createProxyMiddleware({
target,
changeOrigin: true,
pathRewrite: {
"^/api": "", // rewrite path
},
})
);
app.listen(20010);

        

    shell.js

const fs = require("fs");
const shell = require("shelljs");
const { resolve } = require("path");
const order = process.argv.slice(2);
if (!order || order.length === 0) {
shell.echo("pls add target host!");
shell.exit(1);
}

const target = order[0].split("=")[1];
if (!target.startsWith("http")) {
return console.error("target error!");
}
fs.writeFile(
resolve(__dirname, ".TARGET"),
`module.exports = "${target}";`,
(err) => {
if (err) {
return console.error("Err: ", err);
}
const image = "sira-http-proxy";
shell.exec(`docker image rm ${image} -f`)
shell.exec(`docker container stop sira-proxy`);
shell.exec(`docker container rm sira-proxy`);
shell.exec(`docker image build -t ${image} .`);
shell.exec(
`docker container run --name="sira-proxy" -p 10000:20010 -itd ${image} `,
{
async: true,
}
);
}
);


    Dockerfile

FROM node:12.18.4
COPY . /app
WORKDIR /app
RUN npm install
EXPOSE 20010
CMD ["npx", "pm2-runtime", "proxy.js"]


3.3 执行命令

    node shell -t=https://www.tiankele.cn



SQL server 入门笔记

Posted on 2021-04-21 10:09:13
Comments: 0
Author: 可乐小可爱メ
INSERT INTO "tiankele"."dbo"."user" ( "name", "sex", "age") VALUES ( 'zhaoxiaoqi', 2, 28);

alter table [tiankele].[dbo].[user] add id int identity(1,1) not NULL;

DELETE FROM "tiankele"."dbo"."user" ;

CREATE TABLE "user" (
"uid" BIGINT identity(1,1) primary key not null,
"name" VARCHAR(50) NOT NULL,
"sex" INT NOT NULL,
"age" INT NOT NULL
);

SELECT * FROM "tiankele"."dbo"."user" WHERE sex = 1 ORDER BY "uid" DESC OFFSET 0 ROWS FETCH NEXT 1000 ROWS ONLY;


JIRA api上传附件

Posted on 2021-03-21 16:18:02
Comments: 0
Author: 可乐小可爱メ
1. 背景

需要一个支持系统, 面向非JIRA 直接用户, 调用JIRA 方便用户操作。

(此篇以  Issue attachments, add attachement 为例)

2. 设计实现



3. 代码

    3.1 后端部分

        3.1.1  API TOKEN

const { JIRA_TOKEN, JIRA_ACCOUNT } = require("../.secret");
const headers = {
Authorization: `Basic ${Buffer.from(JIRA_ACCOUNT + ":" + JIRA_TOKEN).toString(
"base64"
)}`,
Accept: "application/json",
"Content-Type": "application/json",
};


生成链接 : https://id.atlassian.com/manage-profile/security/api-tokens


        3.1.2 调 JIRA api

// ... 上下文
router.post(`/issue/attachment/:key`, (req, res) => {
const { key } = req.params;
const contentType = req.headers["content-type"];
const form = new Formidable.IncomingForm();
form.parse(req, async (err, fields, files) => {
if (err) {
return res.end({ code: 10001, err });
}
const rs = fs.createReadStream(files.file.path);
let value = [];
rs.on("data", (chunk) => {
value.push(chunk);
});
rs.on("error", (err) => {
res.json({ code: 10001, err });
});
rs.on("end", () => {
request(
{
url: `${JIRA_HOST}/rest/api/3/issue/${key}/attachments`,
method: "POST",
headers: {
Authorization,
Accept: "application/json",
"content-type": contentType,
"X-Atlassian-Token": "no-check",
},
formData: {
file: {
value: Buffer.from(value),
options: {
filename: files.file.name,
contentType: null,
},
},
},
},
(error, response, body) => {
if (error) {
log("jira api err: /issue/comment");
return res.json({ code: 10001, msg: "jira api err" });
}
res.status(response.statusCode);
res.send(body);
}
);
});
});
});

    3.2 前端部分

function uploadFile(e: any) {
const file = new window.File([e.file], e.file.name, { type: e.file.type });
const SIRATOKEN = window.localStorage.getItem("SIRA-TOKEN");
const formData = new FormData();
formData.append("file", file);
axios({
headers: {
Accept: "application/json",
"Content-Type": "multipart/form-data",
SIRATOKEN,
},
url: `/sira/jira/issue/attachment/${selectIssue}`,
method: "POST",
data: formData,
})
.then((res) => {
console.log("res: ", res);
})
.catch((e) => {
console.log("e: ", e);
});
}


4. 注意点

    4.1 这里 node服务,steam流 尝试转成pipe 避免大文件内存不够问题(暂时还没写出来), 需要加上 size 判断拦截.

    4.2 查看文件时,JIRA 文件地址做了 鉴权 302重定向,


并不能简单通过, 

res.set("Authorization", Authorization);
res.redirect(fileUrl);

这样处理来获取,因为真实资源路径是 JIRA 自己再次302而来。

这里的处理是 先 request 获取资源, 其中有当前资源的真实Uri,

router.get(`/file`, (req, res) => {
const {
query: { fileUrl },
} = req;
request(
{
method: "GET",
url: encodeURI(fileUrl),
headers: {
Authorization,
},
},
(error, response, body) => {
if (error) throw new Error(error);
if (response.statusCode === 200) {
res.redirect(response.request.uri.href);
} else {
res.status(response.statusCode);
res.json(response.body);
}
}
);
});


5. 资源链接

https://developer.atlassian.com/cloud/jira/platform/rest/v3/intro/


初探Electron.js

Posted on 2021-02-13 22:05:28
Comments: 1
Author: 可乐小可爱メ
1. 安装

前言介绍就直接跳过了,

按官网 npm i --save-dev electron

但有时候网络环境欠佳就一直安装不成功,(实际上是安装一个 精简版的 Chromium, 默认从github下载,服务器在国外)

即使我nrm 切换到 taobao源指向 


结果也安装超时:


可以尝试 yarn set electron的镜像:

yarn config set ELECTRON_MIRROR https://cdn.npm.taobao.org/dist/electron/

然后: yarn add electron --dev --platform=win64


2. 初始demo项目

这里写一个建议版本的 react 

主进程 index.js

const { app, BrowserWindow } = require("electron");

const createWindow = () => {
const win = new BrowserWindow({
width: 800,
height: 600,
frame: false, // 静止边框和标题栏
webPreferences: {
nodeIntegration: true, // 开放node能力
},
});

win.loadFile("./index.html");
};

app.whenReady().then(createWindow);

app.on("window-all-closed", () => {
if (process.platform !== "darwin") {
app.quit();
}
});

app.on("activate", () => {
if (BrowserWindow.getAllWindows().length === 0) {
createWindow();
}
});


渲染进程 index.html

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>React - ReactRedux</title>
</head>
<body>
<div id="app"></div>
</body>
<script src="https://unpkg.com/react@17/umd/react.development.js"></script>
<script src="https://unpkg.com/react-dom@17/umd/react-dom.development.js"></script>
<script src="https://cdn.bootcss.com/babel-standalone/6.26.0/babel.js"></script>
<script src="https://unpkg.com/redux@4.0.5/dist/redux.js"></script>
<script src="https://unpkg.com/react-redux@7.2.2/dist/react-redux.js"></script>
<script type="text/babel">
const root = document.querySelector("#app");
const reducer = (state = { count: 1 }, action) => {
if (action.type === "PLUS") {
return { count: ++state.count };
} else if (action.type === "REDUCE") {
return { count: --state.count };
}
return state;
};

const store = Redux.createStore(reducer);

const App = (props) => {
const count = ReactRedux.useSelector((state) => state.count);
const dispatch = ReactRedux.useDispatch();
return (
<div>
<p>React - ReactRedux</p>
<button onClick={() => dispatch({ type: "PLUS" })}>+</button>
<p>count: {count}</p>
<button onClick={() => dispatch({ type: "REDUCE" })}>-</button>
</div>
);
};

const ConnectApp = ReactRedux.connect((state) => state)(App);

ReactDOM.render(
<ReactRedux.Provider store={store}>
<ConnectApp />
</ReactRedux.Provider>,
root
);
</script>
</html>


3.  编译

这里是用的 官方维护的 electron-builder

yarn add electron-builder --dev

然后在 package.json 中添加 script 命令

"scripts": {
"dev": "electron .",
"build": "electron-builder --dir",
"test": "echo "Error: no test specified" && exit 1"
},

执行后:



4. 工程化

当然,应该通过 webpack 工程化 引入 React or  Vue

这里给两个传送门, 具体的 electron webpack 及 前端框架的工程化配置 没有做深入研究

electron-react-boilerplate;

Vue的话 官方提供的 @vue/cli 

vue add electron-builder

直接初始话 vue-electron 项目, 通过 package.json 中的 script命令


项目同时支持 web 和 electron 的 开发以及生成模式。

文件转Blob 防盗链

Posted on 2021-01-25 22:16:31
Comments: 0
Author: 可乐小可爱メ
1. 背景

一直想研究下 bilibili 视频 blob防盗链的鬼, 今天做个初探吧


2. 代码

直接上代码吧~~

    2.1 后端部分

const Express = require("express")
const app = new Express()
const fs = require("fs")

app.get("/video", (req, res) => {
const { referer } = req.headers
if (referer !== "http://127.0.0.1:8080/") {
return res.send("404")
}
res.header("Access-Control-Allow-Origin", "*")
const file = fs.readFileSync("./cat.mp4")
res.send(file)
})
app.listen(5010)


    2.2 前端部分

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Blob</title>
<style>
video {
width: 300px;
}
</style>
</head>
<body>
<video src="" id="video" controls></video>
</body>
<script>
const vi = document.getElementById("video")
const xhr = new XMLHttpRequest()
xhr.open("GET", "http://localhost:5010/video", true)
xhr.responseType = "blob"
xhr.onload = function (res) {
if (this.status === 200) {
let blob = this.response
vi.src = URL.createObjectURL(blob)
vi.addEventListener(
"loadedmetadata",
function () {
URL.revokeObjectURL(blob)
},
false
)
}
}
xhr.send()
</script>
</html>


3. 分析

    3.1 暂时用到了 window.URL.createObjectURL ;  即 通过ajax 获取后端的 mp4资源 转换成 blob流.

    3.2 后端部分 读取文件 用 pipe管道 防止内存爆, 同时做 referer 验证, 防止通过抓包 直接下载资源.

    3.3 后续在研究下 是否能直接返回blob地址


4. 参考资料

    为什么视频网站的视频链接地址是blob?

1
2